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ABSTRACT 


A  one-dimensional  (1-D)  approach  to  the  problem  of  adaptive  image  res¬ 
toration  is  presented.  In  this  approach,  we  use  a  cascade  of  four  1-D  adaptive 
filters  oriented  in  the  four  major  correlation  directions  of  the  image,  with  each 
filter  treating  the  image  as  a  1-D  signal.  The  objective  of  this  1-D  approach  is  to 
improve  the  performance  of  the  more  general  two-dimensional  (2-D)  approach. 
This  differs  considerably  from  previous  1-D  approaches,  the  objectives  of  which 
have  typically  been  to  approximate  a  more  general  2-D  approach  for  computa¬ 
tional  reasons  and  not  to  improve  its  performance.  The  main  advantage  of  this 
new  1-D  approach  is  its  capability  to  preserve'  edges  in  the  image  while  remov¬ 
ing  noise  in  all  regions  of  the  image,  including  the  edge  regions.  To  illustrate 
this  point,  the  approach  is  applied  to  existing  2-D  image  restoration  algorithms. 
Experimental  results  with  images  degraded  by  additive  white  noise  at  various 
SNRs  (signal  to  noise  ratios)  are  presented.  Further  examples  illustrate  the 
application  of  1-D  restoration  techniques  based  on  this  approach  to  images 
degraded  by  blurring  and  additive  white  noise  and  images  degraded  by  multipli¬ 
cative  noise.  Another  example  shows  its  usefulness  in  the  reduction  of  quanti¬ 
zation  noise  in  pulse  code  modulation  image  coding.  , 
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INTRODUCTION 


LI  Introduction 

Image  restoration  is  the  reconstruction  of  a  degraded  image  towards 
the  original  object  by  the  reduction  or  removal  of  the  degradations.  These 
degradations  may  be  introduced  during  the  formation,  transmission  and  recep¬ 
tion  of  the  image.  For  example,  an  out-of-focus  camera,  or  the  relative  motion 
between  the  camera  and  the  object,  blurs  the  recorded  picture;  an  image  sensor 
circuit  or  a  transmission  channel  may  introduce  random  noise  to  the  picture; 
an  aerial  photograph  may  suffer  from  distortion  due  to  air  turbulence.  Because 
of  the  recent  advances  in  computer  technology,  digital  image  restoration  has 
received  considerable  attention  for  a  large  number  of  applications,  such  as 
astronomy,  remote  sensing,  medical  imagery,  and  aerial  reconnaissance. 

In  this  thesis,  the  primary  concern  is  the  reduction  of  noise  in  the 
observed  image.  The  noise  reduction  system  is  useful  for  images  which  suffer 
from  little  or  no  blurring.  It  is  also  useful  for  a  class  of  restoration  techniques 
where  the  blurred  and  noisy  image  is  processed  by  a  noise  reduction  system 
prior  to  the  deblurring  process.  Sufficient  noise  reduction  would  improve  the 
performance  of  the  deblurring  process. 

The  performance  of  an  image  restoration  system  depends  on  the  accu¬ 
racy  of  the  model  adopted  for  the  image.  Better  performance  can  be  expected 
if  a  more  accurate  model  is  used.  However,  to  accurately  model  an  image  is  a 
difficult  task,  because  an  image  is  generally  neither  periodic  nor  stationary,  and 


its  formation  cannot  be  characterized  by  a  simple  parametric  model.  As  a 
result,  a  model  that  accurately  describes  an  image  is  likely  to  be  very  complex. 
Even  if  an  accurate  model  could  be  found,  difficulties  could  arise  in  the  estima¬ 
tion  of  the  model  parameters  and  implementation  of  the  restoration  system. 
Due  to  these  difficulties,  a  compromise  is  often  made  between  a  simpler  model 
and  the  performance  of  the  restoration  system. 

In  designing  an  image  restoration  system,  we  must  also  consider  the 
criterion  that  is  used  to  judge  the  performance  of  the  system.  In  many  applica¬ 
tions,  the  restored  images  are  to  be  viewed  by  human  observers.  Therefore,  the 
image  restoration  system  should  be  compatible  with  the  human  visual  system  in 
some  optimum  way.  This  is,  however,  a  difficult  task,  because  of  the  limited 
understanding  of  the  complex  human  vision  system.  Although  one  may  define 
a  measurable  objective  criterion  such  as  the  minimization  of  the  mean  square 
error  between  the  original  and  processed  images,  it  should  not  be  taken  as  an 
absolute  criterion.  For  example,  a  blurred  picture  may  have  a  small  mean 
square  error,  but  it  may  not  be  acceptable  to  human  viewers.  On  ther  other 
hand,  a  more  subjective  criterion,  such  as  the  preservation  of  details  in  the  pic¬ 
ture,  may  be  more  suitable  for  the  human  viewers.  However,  such  a  criterion 
may  not  be  easily  incorporated  in  the  optimization  process. 

Numerous  mathematically  optimal  image  restoration  techniques  have 
been  proposed  [1,2].  The  earlier  techniques  are  mostly  linear  methods,  due  to 
their  simplicity  in  analysis  and  computations.  Based  on  the  assumption  that  the 
signal  and  noise  processes  are  statistically  stationary,  these  linear  methods 
result  in  space  invariant  filters  designed  by  using  some  average  characteristics 
of  the  signal.  Specifically,  for  a  noise  reduction  system,  these  filters  are  some 


form  of  a  low  pass  filter  which  removes  noise  components  at  high  spatial  fre¬ 
quencies.  Unfortunately,  low  pass  filters  also  suppress  the  high  frequency  com¬ 
ponents  of  the  image,  which  convey  important  information  about  the  edges  and 
fine  details  of  the  objects  in  the  image.  The  result  is  an  undesirable  blurring  of 
the  image. 

The  stationarity  assumption  used  in  these  linear  techniques  is  generally 
invalid  in  real  images.  For  example,  the  correlation  between  adjacent  picture 
elements  (pixels)  in  one  part  of  a  picture  with  a  lot  of  spatial  activities  is 
significantly  different  from  the  correlation  in  another  part  which  is  relatively 
flat.  A  low  pass  filter,  based  on  the  signal  characteristics  in  the  flat  areas,  will 
perform  well  in  the  flat  areas  but  will  blur  the  details  in  the  edge  areas. 

Many  adaptive  systems  have  been  proposed  to  overcome  this  difficulty. 
In  these  systems,  a  nonstationary  model  is  assumed,  and  the  model  parameters 
are  updated  with  the  changes  in  image  properties.  The  restoration  filter  is  then 
adapted  to  the  varying  model  parameters.  In  general,  adaptation  could  be 
made  continuously  from  pixel  to  pixel.  However,  this  would  require  a  large 
amount  of  computations.  In  order  to  reduce  computations,  the  adaptation 
model  has  to  be  kept  simple.  An  alternative  approach  is  to  partition  the  image 
into  regions  or  subimages.  Within  each  region  or  subimage,  a  locally  stationary 
model  is  assumed,  and  a  filter  is  designed  with  the  parameters  estimated  for 
that  region.  Although  adaptive  systems  are  computationally  more  expensive  in 
both  design  and  implementation,  they  perform  significantly  better  than  non- 
adaptive  methods. 

Images  are  two-dimensional  (2-D)  signals.  Consequently,  most  image 
restoration  systems  are  2-0  processes.  Images  may  also  be  treated  as  one- 
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dimensional  (1-D)  signals,  so  that  1-D  signal  processing  techniques  may  be 
applied.  A  simple  way  to  characterize  an  image  is  to  consider  it  as  a  collection 
of  1-D  signals,  eg,  the  output  of  a  raster  scanner  [3],  or  a  sequence  of  rows  or 
columns  [4].  1-D  stochastic  models  can  then  be  applied,  and  result  in  1-D  res¬ 
toration  of  the  scanner  output  or  the  individual  rows  or  columns.  This  has  a 
disadvantage  in  that  some  image  correlation  information  is  lost  in  the  conver¬ 
sion  from  the  2-D  domain  to  the  1-D  domain. 

1-D  filters  are  sometimes  used  to  approximate  a  2-D  filter.  In  an  adap¬ 
tive  image  restoration  system,  a  2-D  filter  is  generally  obtained  at  each  pixel  or 
subimage,  using  the  local  image  characteristics  in  the  surrounding  region.  Since 
the  filter  has  to  be  determined  at  every  pixel  or  subimage,  adaptive  systems  are 
computationally  expensive.  In  order  to  reduce  computation,  the  2-D  filter  is 
sometimes  designed  or  implemented  by  several  1-D  filters  as  an  approximation 
to  the  2-D  filter.  This  approach  generally  gives  suboptimal  performance  com¬ 
pared  with  the  original  2-D  filter. 


L2  Scope  of  the  Thesis 

Many  adaptive  image  restoration  systems  apply  a  2-D  spatially  variant 
filter  to  the  degraded  image.  The  filter  is  typically  determined  from  a  small 
local  region  of  the  image  based  on  some  simple  mathematical  criterion  such  as 
the  mean  square  error  minimization.  Within  the  local  region,  the  image  is  usu¬ 
ally  assumed  to  be  a  sample  of  a  stationary  random  process  so  that  methods, 
such  as  Wiener  filtering,  can  be  used  to  determine  the  filter  coefficients.  A 
major  problem  of  this  approach  often  occurs  in  edge  regions  where  the  signal 


cannot  be  adequately  modeled,  even  locally,  as  a  sample  of  a  stationary  random 
process,  and  a  filter  determined  witb  this  assumption  may  not  be  able  to  both 
preserve  edges  and  reduce  noise  at  the  same  time. 

In  this  thesis,  a  new  1-D  approach  to  adaptive  image  restoration  is  pro¬ 
posed  which  aims  to  achieve  better  noise  reduction  near  edges  while  preserving 
the  edges.  The  objective  of  this  approach  is  to  improve  the  performance  of  the 
more  general  2-D  approach.  This  differs  considerably  from  previous  1-D 
approaches,  the  objectives  of  which  have  typically  been  to  approximate  a  more 
general  2-D  approach  for  computational  reasons  and  not  to  improve  its  perfor¬ 
mance.  More  specifically,  restoration  systems  developed  based  on  this  1-D 
approach  remove  noise  more  effectively  than  their  2-D  counterparts,  without 
compromising  the  resolution  of  the  image. 

In  this  approach,  four  1-D  adaptive  filters  oriented  in  the  four  major 
correlation  directions  of  the  image  are  applied  sequentially  to  the  image.  Each 
of  the  four  filters  is  designed  based  on  the  same  image  model  and  basic  princi¬ 
ple  used  in  the  development  of  a  2-D  adaptive  image  restoration  system.  Thus, 
the  noise  at  an  edge  is  removed  by  one  of  the  1-D  filters  oriented  approxi¬ 
mately  in  the  same  direction  as  the  edge.  Since  each  filter  is  adaptive  and  edge 
preserving,  sequential  application  of  the  four  filters  does  not  blur  the  edge. 

The  usefulness  and  limitation  of  this  approach  will  be  studied  by  appli¬ 
cation  to  existing  2-D  image  restoration  methods  for  images  degraded  by  addi¬ 
tive  white  noise.  Although  the  approach  is  developed  primarily  for  the  reduc¬ 
tion  of  noise  in  images,  its  performance  in  a  noise-reduction-deblurring  restora¬ 
tion  system  will  be  studied.  Other  examples  that  are  studied  as  a  part  of  this 
thesis  are:  a)  images  degraded  by  multiplicative  noise,  and  b)  quantization  noise 


reduction  in  a  low  bit  rate  image  coding  system. 

In  summary,  this  thesis  proposes  a  new  1-D  approach  to  adaptive  image 
restoration,  discusses  its  implementation  and  evaluates  its  performance. 

L3  Organization  of  Chapters 

This  thesis  is  organized  in  six  chapters. 

In  Chapter  2,  examples  of  the  common  image  restoration  techniques 
are  reviewed  to  illustrate  the  general  principle  as  well  as  the  limitations  of 
nonadaptive  and  adaptive  approaches  to  image  restoration.  Some  restoration 
systems  using  1-D  approximations  to  2-D  restoration  will  also  be  studied. 

Based  on  the  discussion  in  Chapter  2,  a  new  1-D  image  restoration 
approach  is  developed  in  Chapter  3  for  images  degraded  by  additive  noise.  The 
motivation  and  practical  aspects  of  the  approach  are  discussed. 

In  Chapter  4,  the  1-D  approach  developed  in  Chapter  3  is  applied  to 
three  existing  2-D  adaptive  systems  to  illustrate  its  usefulness  and  limitations. 
The  first  system  is  a  simple  algorithm  based  on  local  statistics  of  the  image  [5]. 
In  the  original  2-D  method,  edges  are  modeled  by  higher  signal  variances,  while 
the  orientation  of  the  edge  is  not  taken  into  account  in  the  model.  This  results 
in  insufficient  noise  removal  near  edges.  Using  the  new  1-D  approach,  the 
modified  system  improves  the  performance  by  reducing  the  noise  near  edges.  In 
the  second  example,  an  adaptive  filter  is  designed  based  on  the  lower  sensitivity 
of  the  human  visual  system  to  noise  near  edges  [6].  Similar  improvement  to  this 
filter  can  be  achieved  with  the  new  1-D  approach.  The  third  system  is  a  short 


space  spectral  subtraction  technique  implemented  in  the  frequency  domain  [7]. 
In  the  2 -D  method,  the  spectrum  of  the  original  image  is  estimated  by  subtract¬ 
ing  the  noise  spectrum  from  the  observed  image  spectrum.  The  1-D  implemen¬ 
tation  of  the  technique  illustrates  a  limitation  of  the  1-D  approach. 

In  Chapter  5,  three  further  applications  of  the  1-D  approach  are  shown. 
The  first  application  is  the  restoration  of  images  degraded  by  blurring  as  well 
as  additive  noise.  Following  an  approach  in  [7],  the  degraded  image  is  processed 
by  a  1-D  noise  reduction  technique  developed  in  Chapter  4,  before  it  is 
deblurred  by  inverse  filtering.  The  noise  reduction  avoids  the  instability  prob¬ 
lem  often  associated  with  inverse  filtering.  The  second  application  demonstrates 
the  effectiveness  of  the  1-D  technique  in  the  density  (log  intensity)  domain  to 
restore  images  corrupted  by  multiplicative  noise.  In  the  third  application,  the 
1-D  method  is  used  as  a  post-processor  for  a  particular  bit  rate  reduction 
scheme  known  as  the  Roberts’  pseudonoise  method  [9].  In  this  scheme,  the 
signal-dependent  quantization  noise  associated  with  low  bit  rate  quantization  is 
converted  to  less  objectionable  signal-independent  random  noise.  The  1-D 
method  improves  the  appearance  of  the  image  by  reducing  the  amount  of  ran¬ 
dom  noise. 

In  the  final  chapter,  a  summary  of  results  is  given  along  with  sugges¬ 
tions  for  future  research  using  the  1-D  approach. 
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CHAPTER  2 

REVIEW  OF  IMAGE  RESTORATION 

2.1  Introduction 

Numerous  methods  have  been  proposed  to  solve  the  problem  of  image 
restoration.  Some  are  ad-hoc;  others  are  solutions  to  some  optimization  prob¬ 
lems.  There  are  various  ways  to  categorize  these  techniques,  according  to  the 
assumptions  made  about  the  image,  the  restoration  criterion,  and  their  imple¬ 
mentation.  For  the  purpose  of  this  thesis,  they  will  be  classified  as  either  nona- 
daptive  or  adaptive.  The  major  difference  between  these  two  categories  lies  in 
the  assumptions  made  about  the  image  model:  a  stationary  model  generally 
leads  to  nonadaptive  techniques,  and  a  nonstationary  model  leads  to  adaptive 
techniques. 

Some  examples  of  both  nonadaptive  and  adaptive  image  restoration 
will  be  reviewed  to  illustrate  the  general  assumptions,  principles  and  limitations 
of  the  two  approaches.  Although  adaptive  processing  generally  performs  better 
than  nonadaptive  processing,  it  incurs  heavy  computational  loads.  Sometimes, 
the  computational  loads  may  be  reduced  by  using  1-D  approximations  to  the  2- 
D  adaptive  approach.  The  general  principle  of  these  approximations  will  be  stu- 
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22  Representations  of  Images  and  Degradations 

Depending  upon  the  specific  application  and  processing  technique 
employed,  an  image  may  be  represented  in  several  different  ways,  eg,  space 
domain  versus  frequency  domain  representation;  2-D  versus  1-D  representation. 
In  this  section,  the  various  representations  will  be  discussed,  followed  by  a 
description  of  the  commonly  used  image  and  degradation  models. 

Image  Representation 

Let  g  (x  ,y )  be  a  continuous  image  in  the  image  plane  (x  ,y ).  To 
represent  the  image  in  the  discrete  form  for  digital  processing,  it  is  necessary  to 
sample  the  continuous  image  according  to  the  two-dimensional  sampling 
theorem.  Thus,  if  the  Nyquist  criterion  is  satisfied,  the  image  may  be 
represented  by  the  2-D  sequence  g(n1,n2)  where  and  n2  are,  respectively,  the 
distance  indices  in  the  horizontal  and  vertical  directions. 

Another  discrete  representation  of  the  image  is  obtained  by  raster 
scanning  the  image  to  form  a  1-D  vector  g,  where 

*(U) 

*(1,2) 

g=  *(W  (2-1) 

*(2,1) 

*  (N  yN  ) 

where  N  is  the  size  of  *(«i,n2).  This  representation  is  most  useful  in  linear 
algebraic  restoration  methods. 


In  the  frequency  domain,  g(»i,n2 )  is  represented  by  its  Fourier 


transform  defined  as 

=  S'  l'  S(nb»2)  (22) 

<«l=o  /ijm 

This  representation  is  useful  for  processing  in  the  frequency  domain. 

One  useful  representation  often  used  in  recursive  algorithms  is  that  of 
state  space  representation.  It  may  take  many  different  forms  depending  on  the 
state  variables  chosen.  In  state  space  representation,  the  image  is  modeled  as 
the  output  of  a  dynamical  system,  which  relates  the  present  state  at  a  pixel  of 
the  image  to  the  states  of  its  neighbors.  A  general  form  of  the  dynamical 
model  is  given  by 


s(n  )  =  A(/i )  s(n  -1)  +  B(n  )  u(n  ) 

(23) 

r(n  )  =  C(n  )  s(n  )  +  w(n  ) 

(2.4) 

where  s  is  the  state  vector,  r  is  the  observation,  w  is  the  observation  noise,  and 
u  is  the  white  noise  that  drives  the  system.  A,  B  and  C  are,  respectively,  the 
system,  drive  and  observation  matrices.  State  space  representation  is  typically 
used  in  the  causal  estimation  of  a  future  pixel  from  the  noisy  observation  and 
some  amount  of  information  about  past  and  present  estimates. 

Image  Formation 

To  restore  images  suffering  from  spatial  degradation,  such  as  blurring, 
the  degradation  must  be  characterized  mathematically.  Let  g{n^,n2)  be  an 
image  of  an  object  f  (ky,k2).  In  practical  imaging  systems,  the  intensity  of  the 
image  at  the  point  (ni,n2),g(ni,n2),  is  a  function  of  /  (kuk2)-  If  the  function  is 
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linear,  the  general  description  of  g  is 

g(n\,n2)  =X  X  h{nx,n2,kx,k2)  f  ikx,k2)  (25) 

*.  *z 

The  function  h  is  referred  to  as  the  point  spread  function  (PSF).  If  h  is 
assumed  space  invariant, 

gin hn2)  =2  X  hin\~k\ »«2“*2)/  (*l»*2> 

i,  i, 

=  A(n1,fl2)  */(«!,«  2)  (25) 


where  *  denotes  2-D  convolution.  In  terms  of  vector -matrix  representation, 

g=Hf  (2.7) 

where  g  and  f  are  vector  created  in  the  manner  described  by  Eqn.  (2d),  and  H 
is  a  N  2  xN  2  matrix  created  from  h  (n  \,n2)- 

In  the  frequency  domain,  Eqn.  (25)  can  be  written,  by  taking  Fourier 
Transforms  on  both  sides,  as 

G  (011,(1^)  =  H  (<•>!, {*>2)  F  (o>i,<«»2)  (23) 

where  G  (<>>], o^),  F  (tu^o^)  and  H  (co^u^)  are  the  Fourier  transforms  of  g(«i,n2), 
/  in\yn2),  and  h(nt,n2)  respectively. 

Point  Spread  Function 

The  PSF  h(nun2;kuk2)  is  the  response  of  the  imaging  system  to  a  unit 

point  source  located  at  (ki,k2).  It  can  be  used  to  characterize  many  distortion 

and  blurring  effects  of  the  imaging  system.  One  such  example  is  the  long  term 
exposure  in  atmospheric  turbulence,  where  the  PSF  is  approximately  Gaussian 


0 


and  given  by 


_  l 

H{  o,lfu»2)=e  2  •*  (29) 

where  a2  determines  the  degree  of  spreading. 

Image  Model 

An  image  is  described  by  a  model  which  is  an  abstraction  of  the  struc¬ 
ture  of  the  image  or  of  a  class  of  images.  One  commonly  used  model  in  image 
restoration  is  a  statistical  model.  An  image  is  then  a  sample  of  the  random 
process  described  by  the  model. 

If  the  probability  density  function  (PDF)  of  the  image  process  is 
known,  the  parameters  of  the  PDF  completely  describe  the  image.  One  exam¬ 
ple  is  the  multivariate  Gaussian  PDF,  which  is  fully  characterized  by  its  mean 
vector  f  and  covariance  matrix  defined,  respectively,  by 

f  «  £  [f  ]  (2  JO) 

and 

=£[(f-f)(f-f)r]  (211) 

where  E  [  J  denotes  expectation.  In  most  cases,  the  specific  PDF  is  not  known. 
A  good,  concise  description  of  the  image  can  still  be  given  by  the  mean  f  and 
the  covariance  function  . 

A  common  assumption  on  image  model  is  that  of  stationarity.  A  ran¬ 
dom  process  is  said  to  be  strict-sense  stationary  (SSS)  if  the  joint  PDF  of  any 
collection  of  samples  depends  only  upon  the  spacing  between  samples.  Because 
it  is  usually  difficult  to  show  that  a  given  process  is  SSS,  a  more  practical  and 
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more  common  assumption  of  stationarity  is  that  of  wide-sense  stationarity.  An 
image  is  wide-sense  stationary  if  the  mean  is  constant  and  the  correlation 
between  pixels  is  independent  of  the  absolute  locations  of  the  pixels,  i n, 

f(n  xji2)  ~  E  \f  ("  t»«2)]  =  constant  (212) 

R(nhn2;khk2)  =E[f  (nhn2)f  (kx,k2)]  =& (ni~kun2-k2)  (213) 

In  this  case,  as  defined  in  Eqn.  (211)  has  a  block  Toeplitz  structure. 

Noise  Model 

A  noise  model  commonly  assumed  in  image  restoration  is  that  of  addi¬ 
tive  white  noise.  A  degraded  noisy  image  may  then  be  described  by 

*(«l»«2)  =/(«l>*2)*  h(nx,n2)+w(nx,n2)  (214) 

where  w(n1,n2)  is  a  signal-independent  white  noise  sequence.  Many  physical 
noise  processes  can  be  approximated  as  white'  noise,  eg.,  random  thermal  noise 
in  image  sensor  circuits. 

Another  common  noise  model  is  that  of  multiplicative  noise.  It  is 
described  by 

8(nx,nz)  =  [f  (n\,n2)*h(nx,n2)\v(nXtn2)  (215) 

One  example  of  multiplicative  noise  is  the  speckle  noise  in  laser  generated 
images.  In  general,  if  an  additive  noise  component  is  proportional  to  the  signal, 
it  can  also  be  modeled  as  multiplicative  noise. 


23  Nonadaptive  Image  Restoration 

In  a  stationary  image  model,  an  image  is  assumed  to  be  a  sample  of  a 
wide-sense  stationary  random  process  with  a  constant  mean  vector  and  a  block 
Toeplitz  covariance  matrix.  Linear  filtering,  generally  based  on  some  least 
square  criteria,  can  then  be  applied  to  obtain  an  estimate  of  the  original  image 
from  the  degraded  observed  image.  The  filter  may  be  implemented  in  the  fre¬ 
quency  domain  to  take  advantage  of  the  computational  efficiency  of  the  FFT 
(Fast  Fourier  Transform)  algorithm.  In  the  space  domain,  using  state  space 
representation  and  a  causal  model,  recursive  algorithms  also  result  in  efficient 
implementation.  Two  typical  examples  of  linear  filtering  will  be  discussed. 

Wiener  Filter 

The  following  model  is  assumed: 

g  =  Hf  +  w  (216) 


where  f  is  assumed  zero-mean  and  w  is  signal  independent.  It  is  also  assumed 
that  the  image  correlation  and  the  noise  process  are  known  a  priori.  By  impos¬ 
ing  a  minimum-mean -square-error  criterion,  which  minimizes  the  expected 
error  f  -f  of  the  estimate  f  over  the  ensemble  of  all  possible  images,  ie.. 


min  £  ((  f  -f  )T  ( f  -f  )] 

A 

f 


(217) 


the  estimate  was  derived  by  Helstrom  [11]  as 

f  =  *>fHT  (HQf  Ht  +4>wr1g  (218) 


where  <1 and  4>w  are  the  autocovariance  matrices  of  f  and  w  respectively 


Because  of  the  high  dimensionality  of  the  matrices  ( N1xN 2),  Eqn. 
(218)  cannot  be  evaluated  efficiently.  However,  if  f  and  w  are  stationary 
processes,  then  and  are  block  Toeplitz,  and  Eqn.  (218)  can  be  evaluated 
by  DFT  (Discrete  Fourier  Transform)  approximation  as 


F(khk2)  = 


H'(khk2) 


\H(khk2)\2  + 


M*  i,*2) 

Pf  (k  itk  2) 


G(*„*2) 


(219) 


where  F{kx,k2),  G(klfk 2),  and  H(kt,k2 )  are  the  DFT’s  of  f  (nun2),  g(nhn2), 
and  h(nr,n2)  respectively.  P/(ki,k2)  and  Pw(k\,k2)  are  the  discrete  samples  of 
the  a  priori  power  spectra  of  f  (n\,n2)  and  w(«1,n2)  respectively.  Thus,  if 
H  (Jk  1,Jk  2)  is  als°  known  a  priori,  f  (n\,n2)  may  be  obtained  by  the  inverse  DFT 
of  F  (khk2)  calculated  from  Eqn.  (219). 

2-D  Kalman  filter 

The  application  of  recursive  methods  in  image  restoration  is  motivated 
by  the  success  of  Kalman  filtering  in  1-D  filtering  and  prediction  problems. 
With  the  techniques  of  spectral  factorization,  the  dynamic  model  of  the  image 
random  field  may  be  described  by  a  partial  difference  equation  that  is  recur¬ 
sively  computable.  The  direct  extension  of  the  Kalman  filter  to  image  restora¬ 
tion  was  first  studied  by  Habibi  [12].  In  his  approach,  Habibi  assumed  that  the 
random  field  is  a  separable  2-D  first  order  Markov  process  whose  autocorrela¬ 
tion  function  is  given  by 

*(*!>* 2)  •a}e~9ilkl'e~9t,k2t  (220) 


The  image  can  then  be  represented  as  the  output  of  the  following  autoregres 
sive  system 
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/  (n1+l,«2+l)  =  Pi/  («i+l,/»2)  +  P2/  (ni»*2+l)  “  P1P2 f  (^i»^2> 

(221) 

where  the  correlation  coefficients  pi,  P2  are  given  respectively  by 

Pl=«“"  (222) 

and 

p2=e"ai  (223) 

and  u(n1>n2)  is  a  white  noise  sequence. 

The  minimum  mean  square  error  estimate  of  /  is  given  recursively  by 

/(n1+l,n2+l)  ssPl/(»l+l.«2)  +P2/('ll»"2+l) 

-KPiP 2+tf  («l.»2)]/(»l»«2)  +  AT  (nhn2)y  ( nltn2 )  (224) 

where  AT  (n1,n2)>  a  spatially  varying  function  of  g ,  is  also  recursively  comput¬ 
able.  Thus,  if  at  and  a2  are  known,  Eqn.  (224)  recursively  generates  all  values 
of  /,  given  assumed  boundary  conditions  at  the  topmost  row  and  leftmost 
column  of  the  image. 

Discussion 

In  these  examples,  as  well  as  in  their  other  extensions,  the  autocorrela¬ 
tion  function  of  the  original  image  (or  equivalently,  its  power  spectrum)  must 
be  known  a  priori.  This  is  not  true  in  most  practical  situations.  The  autocorre¬ 
lation  function  can  also  be  estimated  by  some  other  means,  for  example,  by 
averaging  the  autocorrelation  functions  of  several  prototype  images.  However, 
the  accuracy  of  this  approach  is  questionable.  These  techniques  also  require 
that  the  image  be  stationary  to  achieve  efficient  algorithms.  In  the  Wiener 


filter,  for  example,  nonstationary  correlation  matrices  make  fast  implementa¬ 
tion  in  the  frequency  domain  impossible.  In  the  2-D  Kalman  filter,  although 
nonstationarity  may  be  accounted  for  by  spatially  varying  correlation  func¬ 
tions,  estimating  them  in  the  noisy  image  remains  a  difficult  task. 

Pictures  processed  by  nonadaptive  techniques  generally  appear  to  be 
too  smooth.  This  is  partly  because  the  solutions  are  obtained  by  a  minimum 
mean  square  error  criterion,  which  is  not  the  criterion  the  human  visual  system 
uses  to  judge  the  quality  of  an  image.  More  importantly,  the  smoothness  is 
caused  by  the  simple  stationary  assumption  which  is  invalid  in  most  images. 

2A  Adaptive  Image  Restoration 

The  previous  section  illustrated  the  shortcomings  of  nonadaptive  linear 
restoration  procedures.  The  main  difficulty  was  due  to  the  assumption  of  sta- 
tionarity.  The  general  approach  to  solving  this  problem  is  to  perform  subimage 
processing,  in  which  the  image  is  first  divided  into  subimages.  Each  subimage  is 
then  processed  independently,  assuming  local  stationarity.  In  this  case,  the 
model  parameters  have  to  be  determined  for  every  subimage.  In  order  to 
reduce  the  amount  of  calculation,  the  model  is  often  a  simple  one,  as  illustrated 
by  the  two  following  examples.  The  first  example  deals  with  nonlinear  sensor 
characteristics.  The  second  example  is  an  adaptive  Kalman  filter. 

Sectioned  MAP  method 

Trussell  and  Hunt  [13]  derived  a  maximum  a  posteriori  (MAP)  method 
for  restoring  images  degraded  by  nonlinear  sensor  characteristics,  as  described 
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g  =  s  [Hf  ]  +  w 


(225) 


where  r[-]  represents  the  nonlinearity.  Using  the  multivariate  Gaussian 


assumption  for  f  and  w,the  solution  is  given  by  an  implicit  equation  : 


f  =f  +<D/HrS*4>-1[g^[Hf] 


(226) 


where  f  is  the  a  priori  mean  of  the  original  image  and  Sb  is  a  diagonal  matrix 
of  derivatives  of  s . 

By  dividing  the  image  into  blocks,  Eqn.  (226)  is  solved  iteratively  for 
the  estimate  in  each  block.  To  simplify  the  computation,  and  are 
replaced  by  aj  and  respectively,  ie.,  the  spatial  correlation  is  ignored.  The 


estimate  for  block  m  is,  therefore, 


f„  =r.  +-^-HrsJg. 


(227) 


The  results  are  combined  by  the  overlap  save  method. 

Adaptive  Kalman  Filter 

Instead  of  dividing  the  image  into  fixed-sized  blocks,  the  partitioning 
can  also  be  done  by  other  means.  For  example,  the  image  can  be  partitioned 
into  regions  according  to  local  correlation  characteristics.  Then,  a  filter  using 
the  correlation  parameters  in  each  region  can  be  implemented.  In  [14],  parti¬ 
tioning  is  done  by  thresholding  the  spatial  activities  at  each  pixel,  measured  in 
terms  of  signal  directional  slope  information,  into  one  of  Af  values.  Each 
region  is  then  characterized  by  one  of  M  stationary  autocorrelation  functions 
which,  for  simplicity,  are  assumed  to  be  exponential  and  separable.  The 


correlation  coefficients  pj,  pi  are  determined  for  each  of  the  autocorrelation 
functions.  A  bank  of  Kalman  filters  are  then  run  in  parallel,  one  for  each 
region,  with  special  care  to  the  boundary  conditions  at  the  border  between 
regions. 

Discussiod 

In  most  adaptive  restoration  systems,  the  algorithm  implemented  in  a 
local  region  is  generally  a  straightforward  extension  of  the  global  nonadaptive 
algorithm.  Since  local  signal  characteristics  must  be  determined  for  each 
subimage  or  region,  the  image  model  for  each  subimage  or  region  has  to  be 
kept  simple  to  reduce  the  amount  of  computation  in  both  model  parameter 
identification  and  filter  implementation. 

Partitioning  of  the  image  may  be  done  block  by  block  or  by  variable 
size  and  shape.  Since  each  subimage  is  processed  by  a  filter  which  may  be 
significantly  different  from  the  ones  in  the  adjacent  subimages,  care  must  be 
taken  in  combining  the  results  of  neighboring  subimages,  as  well  as  in  matching 
boundary  conditions  at  the  border. 

2.5  One-Dimensional  Approximations  to  Two-Dimensional  Image  Restoration 

In  this  section,  we  will  summarize  some  restoration  techniques  using  1- 
D  processing.  Such  1-D  processing  is  typically  used  as  an  approximation  to  the 
2-D  approach,  partly  because  of  the  difficulty  in  establishing  a  good  2-D  image 
model,  and  partly  to  reduce  the  heavy  computational  loads  associated  with 
adaptive  filtering. 
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Restoration  Based  on  a  Multiple  Fragment  Model 

In  [15],  Lebedev  and  Mirkin  assumed  that  an  image  is  composed  of  five 
classes  of  fragments.  Each  class  of  fragments  is  distinguished  by  the  type  of 
correlational  links  between  the  elements  and  characterized  by  a  specific  corre¬ 
lation  Rg  (1:S0<5).  Four  of  the  classes  correspond  to  prominent  correlational 
links  in  the  four  directions  at  0°,  45°,  90°  and  135°.  The  fifth  describes  fragments 
with  an  isotropic  structure.  It  is  assumed  that  p  (0),  the  a  priori  probability 
density  of  the  five  classes,  is  known.  For  each  class  0,p(g  10),  the  conditional 
probability  density  of  the  image  data,  is  assumed  Gaussian  and  known. 

A 

The  estimate  of  /  in  a  window  is  calculated  as  follows.  Let  /  (ni,n2l0) 
be  the  conditional  estimate  of  /  assuming  that  the  windowed  data  belong  to 
the  particular  fragment  class  0.  For  each  0,  since  Ra  is  assumed  known, 
f(nhn 2 1 0)  can  be  calculated  by  linear  methods,  for  example,  Wiener  filter.  Let 
p(0lg)  be  the  a  posteriori  probability  that  the  windowed  data  belong  to  the 


class  0.  p  (01# )  can  be  obtained  by  Bayes  rule  as 


(0)  p  (*  ie) 


(228) 


The  estimate  of  /  is  then  the  sum  of  the  conditional  estimate  /(ni,n2IO), 


weighted  by  p(0l$),  ie. 


/(» i»«2>  =  2  P(0lf)/("i»"21®) 
8=1 


(229) 


In  summary,  the  image  data  in  a  moving  window  are  filtered  by  four 
1-0  filters  and  a  2-D  averaging  filter.  The  restored  image  is  given  by  the  sum  of 
the  five  outputs  with  weights  equal  top(0lg). 


Modified  Wiener  Filter 


Abramatic  and  Silverman  [16]  derived  two  spatially  variant  filters  for 
restoring  unblurred  but  noisy  images.  The  filters  are  Wiener  filters  modified  by 
a  visibility  function  F(n1,n2)  proposed  by  Anderson  and  Netravali  [6].  The 
transfer  functions  of  the  two  filters  are 


and 


Hr  (ta\>(a2‘>n\>n2) 


Pf(o>h<l>2;nl,n2) 

P/(o>l,o»2;n1,n2)  +F(nhn2)oZ 


(230) 


Hg(u>ha>2;nun2)=F(nltn2)  +  [1-F (i»„n2)]  (231) 

P/(o»1,a»2;n1,n2) 


where  Pf  (o)t,o>2;n1,n2)  is  the  local  power  spectrum  of  /  at  {n\,n2).  The  noise 
is  assumed  additive  and  white  with  power  a*. 

The  sequence  F  (nltn2),  with  values  between  0  and  1,  is  a  monotoni- 
cally  decreasing  function  of  the  'busyness'  of  the  pixel  at  (n1,n2)-  As  F  (nun2) 
varies  from  0  to  1,  the  filters  are  a  compromise  between  the  identity  filter  and 
the  Wiener  filter.  Thus,  in  edge  regions  where  F  (n\,n2)  is  close  to  0,  the  filters 
approach  the  identity  filter,  and  edges  are  preserved. 

Since  Pf  (&>11,a»2;nl,n2)  has  to  be  estimated  at  every  point  (n  i,n2),  two 
suboptimal  procedures  were  proposed.  In  both  procedures,  the  image  is 
described  by  a  single  known  power  spectrum.  Its  nonstationary  properties  are 
contained  only  in  the  space  varying  visibility  function  F  (nhn2).  In  the  first  pro¬ 
cedure,  both  the  power  spectrum  and  the  restoration  filters  are  assumed  separ¬ 
able  so  that  Hg  (on,o»2;ni,n2)  can  be  approximated  by  a  horizontal  filter 
H  i(wi;n1,n2)  and  a  vertical  filter  2(0>2i/» 2)- 1°  tbe  second  procedure,  a  1-D 
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filter  calculated  by  either  Eqn.  (230)  or  Eqn.  (231)  is  determined  for  each  of 
the  4  directions  at  0°,  45°,  90°,  and  135°.  The  approximate  2-D  filter  is  recon¬ 
structed  by  bilinearly  interpolating  the  four  filters. 

Order  Constrained  Median  Filter 

In  [17],  the  estimate  at  a  pixel  is  given  by  the  average  of  the  outputs  of 
two  1-D  filters  oriented  in  the  horizontal  and  vertical  directions.  Within  each 
filter  window,  the  filtering  process  consists  of  two  steps.  The  first  step  is  a 
trend  detection  scheme  based  on  a  simple  hypothesis  test.  Assuming  that  the  a 

priori  means  of  the  image  data  in  the  window  are  m  1,012, . /n„,  the  trend  of 

the  data,  assumed  to  be  either  monotonically  increasing  or  monotonically 


decreasing,  is  determined  by  the  hypothesis  test: 

. ^ mn  (232) 

versus 

H  i :  m1arm2  ^ . —mn  (233) 


Under  each  hypothesis,  the  maximum  likelihood  estimate  of  m,- ,  constrained  by 
the  assumed  trend  of  the  data,  is  determined.  In  step  two,  the  median  of  the 
estimate  ifl, ,  obtained  under  the  selected  hypothesis,  is  taken  as  the  output  of 
the  1-D  filter. 

Discussion 

Approximations  to  2-D  restoration  filter  arise  from  the  difficulty  in 
obtaining  the  local  edge  information  or  the  local  power  spectrum  from  the 
degraded  image.  As  illustrated  by  the  examples,  one  way  to  approximate  a  2-D 
filter  is  to  assume  that  the  filter  is  separable.  This  leads  to  cascaded  1-D  filters 


/ 


$ 


in  the  horizontal  and  vertical  directions.  A  more  common  approach  is  to  sum, 
average,  or  select  the  outputs  of  several  1-D  filters  running  in  parallel.  These 
approximations  generally  give  suboptimal  performance  compared  to  the  truly 
2-D  approach,  although  the  amount  of  computation  is  reduced. 

2.6  Summary 

This  chapter  reviewed  the  common  approaches  to  image  restoration. 
The  earlier  approach  based  on  linear  filtering  is  not  suited  to  the  nonstationary 
nature  of  practical  images.  Adaptive  restoration  by  subimage  processing  has 
been  the  general  approach  adopted  recently.  One  disadvantage  of  sue.  an 
approach  is  the  amount  of  computation  required.  Primarily  to  reduce  compu¬ 
tation,  several  1-D  processing  schemes  have  been  proposed  as  an  approximation 
to  the  2-D  adaptive  filter.  However,  such  an  approximation  would  result  in 
suboptimal  performance.  In  the  next  chapter,  a  new  1-D  approach  to  adaptive 
image  restoration  will  be  presented. 
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CHAPTER  3 

A  ONE-DIMENSIONAL  APPROACH  TO  ADAPTIVE  IMAGE  RESTORATION 


3.1  Introduction 

From  the  brief  review  in  chapter  2,  one  can  conclude  that  a  good 
image  restoration  system 

(1)  should  be  based  on  a  nonstationary  image  model  and  able  to 
adapt  to  the  changing  characteristics  of  an  image, 

(2)  should  require  little  a  priori  information  about  the  original 
image  and  estimate  the  filter  parameters  from  the  observed 
image, 

(3)  should  be  easy  to  implement  and  require  minimal  computa¬ 
tional  efforts,  and 

(4)  should  reduce  noise  while  preserving  edges  and  details  in  the 
image. 

Many  existing  restoration  methods  possess  these  properties,  but  their 
performance  has  rooms  for  improvement.  More  specifically,  some  of  these 
methods  are  bused  on  simple  nonstationary  image  models,  and  preserve  edges 
by  removing  less  noise  in  the  edge  areas  than  in  the  fiat  areas.  As  a  result,  the 
edge  areas  appear  noisy,  especially  for  images  at  low  signal  to  noise  ratios 
(SNR).  In  this  chapter,  a  new  approach  making  use  of  1-D  filters  is  proposed. 
The  objective  of  this  approach  is  to  improve  the  performance  of  the  general  2- 
D  approach  for  some  adaptive  image  restoration  systems. 


3 2  Motivation  of  the  Approach 

Ideally,  the  desired  filter  that  would  remove  noise  in  all  regions, 
including  the  edge  regions,  should  have  a  large  support  in  fiat  areas  for  better 
noise  smoothing.  In  the  edge  regions,  the  filter  should  be  nonisotropic  and 
change  its  shape  and  size  to  adapt  to  the  edges  in  order  to  reduce  noise  at  the 
edges  and  to  preserve  the  edges. 

Examination  of  some  adaptive  restoration  techniques  [eg.,  5,6]  reveals 
that  the  filters  are 

(1)  based  on  a  simple  image  model  which  is  insensitive  to  the 
orientation  of  the  edges, 

(2)  of  the  same  support  size  regardless  of  the  image  characteris¬ 
tics,  and 

(3)  isotropic,  in.,  symmetrical  in  all  directions  about  the  original. 
As  a  consequence,  the  performance  of  these  filters  is  not  satisfactory. 

One  approach  to  achieve  better  noise  reduction  and  edge  preservation 
is  to  model  the  image  more  accurately  (considering  an  image  edge  as  a  deter¬ 
ministic  component,  for  example)  and  develop  a  new  image  restoration  algo¬ 
rithm  based  on  the  new  model.  However,  modeling  an  image  accurately  is  a 
difficult  task,  and  the  resulting  image  restoration  algorithm  based  on  a  detailed 
accurate  image  model  is  likely  to  be  quite  complex. 

A  much  simpler  approach  is  to  use  a  1-D  filter  without  changing  the 
image  model  or  the  basic  principle  used  in  developing  a  2-D  image  restoration 
system.  If  a  1-D  filter  is  allowed  to  change  its  orientation,  it  may  be  aligned  in 
the  same  direction  as  the  edge,  and  avoid  intersecting  the  edge.  In  this  manner, 
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the  noise  at  the  edge  is  removed  and  the  edge  is  preserved.  The  next  section 
presents  a  1-D  approach  to  image  restoration  which  is  simple,  does  not  require 
an  explicit  edge  detector  to  orient  a  1-D  filter,  and  can  be  applied  to  both  edge 
and  non-edge  regions.  As  will  be  discussed  in  the  next  chapter,  when  the  1-D 
approach  is  applied  to  some  existing  2-D  image  restoration  algorithms,  tech¬ 
niques  developed  based  on  the  1-D  approach  performs  significantly  better  than 
the  corresponding  2-D  techniques. 


3J  The  Basic  Principle 

Consider  an  image  /  («i,«2)  degraded  by  some  noise  H'(n1,n2).  The 
degraded  image  will  be  denoted  by  g  If  a  2-D  adaptive  filter  is 

represented  as  T[  ],  /(«i,n2)*  the  output  of  the  adaptive  filter,  can  be 
represented  as 

/  (» 2)  =T[g(n\>n  2)1  (3  J) 

The  adaptive  filter  T  [  •  J  is  typically  determined  from  a  local  2-D  region  (or 
window)  of  the  image  surrounding  (n  \,n  2)  based  on  some  image  restoration 
principle,  such  as  Wiener  filtering. 

Let  *]>  (l  —  1  —N),  represent  the  1-D  filter,  which  is  obtained  in  the 
same  way  as  the  2-D  adaptive  filter,  but  determined  from  a  local  1-D  region  (or 
window)  oriented  in  the  i,h  direction.  For  practical  reasons,  let  N  =4,  and  let 
the  four  directions  be  the  major  correlation  directions  of  most  practical  images, 
ie.,  0,  90,  45  and  135  degrees.  The  noisy  image  is  filtered  by  a  cascade  of  the 
four  filters,  as  if  the  image  were  a  1-D  signal  for  each  of  the  filters  (Figure  3 J). 
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The  output  of  the  system  /  (rt^/12)  is  then  given  by 


/('»!>«  2)  =r4 


^  i[«  (»i,n2)  ] 


(32) 


It  should  be  noted  that  7*{  ’s  are  not  obtained  by  factoring  T .  If  they 
were,  the  system  of  the  cascaded  1-D  filters  would  be  identical  to  T .  The 
approach  also  does  not  attempt  to  approximate  T  by  a  cascade  of  7*,-  ’s.  The 
output  of  the  cascaded  filters  may  therefore  be  significantly  different  from  the 
output  of  the  2-D  filter. 


3.4  Discussion 

3.4.1  Advantages  of  the  approach 

The  cascaded  I-D  processing  improves  the  existing  2-D  methods 
without  modifying  the  assumed  image  model  or  the  basic  principle  of  the  2-D 
methods.  Specifically, 

(1)  in  the  flat  areas,  the  cascaded  1-D  filters  is  equivalent  to  a  sin¬ 
gle  2-D  filter  with  a  much  larger  support  size  than  the  original 
2-D  filter,  thus  resulting  in  more  effective  noise  reduction; 

(2)  in  the  edge  regions,  the  noise  near  an  edge  is  smoothed  by  one 
of  the  1-D  filters  oriented  approximately  in  the  same  direction 
as  the  edge; 

(3)  since  the  1-D  filters  are  adaptive  and  edge  preserving,  sequen¬ 
tial  application  of  the  four  filters  does  not  blur  the  edges; 
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(4)  the  computational  load  of  this  approach  is  typically  lighter 
than  the  corresponding  2-D  filtering  because  of  the  1-D  calcu¬ 
lations. 

3A2  Comparison  with  Other  1-D  Approaches 

A  few  examples  of  other  1-D  approaches  to  adaptive  image  restoration 
have  been  reviewed  in  the  previous  chapter.  Their  general  principle  can  be 
summarized  by  Figure  32.  The  noisy  image  is  filtered  by  a  bank  of  filters  whose 
outputs  are  summed,  selected,  or  averaged  to  give  the  output  of  the  overall  sys¬ 
tem. 

The  parallel  structure  of  these  approaches  is  obviously  different  from 
the  cascade  structure  of  the  present  approach.  More  importantly,  their  objec¬ 
tives  in  using  1-D  filters  are  fundamentally  different.  The  other  1-D  approaches 
generally  aim  to  reduce  the  amount  of  computations  by  approximating  the 
truly  2-D  filters.  The  results  are  generally  suboptimal  compared  to  the  truly  2- 
D  filters,  as  some  information  in  the  2-D  domain  is  unavoidably  lost  in  the 
approximation.  In  contrast,  the  goal  of  the  present  approach  is  to  improve  the 
performance  of  the  2-D  filter  by  using  1-D  filters  to  compensate  for  the  lack  of 
edge  information  in  the  2-D  filter. 

Since  the  present  approach  has  a  cascade  structure,  weight  calculations 
are  not  necessary.  In  the  other  approaches,  the  manner  in  which  the  results  are 
combined  may  result  in  further  deterioration  of  performance.  For  example, 
any  forms  of  unweighted  or  weighted  averaging  would  blur  the  edges,  while 
selecting  one  output  out  of  several  would  result  in  insufficient  noise  smoothing. 


filter  1 


3.4.3  Practical  Considerations 

Because  the  filterings  are  performed  sequentially,  the  signal  and  noise 
characteristics  change  after  each  filtering  and  have  to  be  updated  before  subse¬ 
quent  filtering.  Certain  assumptions  may  have  to  be  made  so  that  the  same  res¬ 
toration  principle  may  be  used  on  all  four  filters.  For  example,  if  the  principle 
is  based  on  the  assumption  that  the  image  and  noise  are  not  correlated,  the 
same  has  to  be  assumed  for  all  four  filters. 

Although  four  sequential  filtering  operations  are  involved,  a  large  on¬ 
line  memory  is  not  required.  Since  each  of  the  1-D  filters  performs  local  filter¬ 
ing,  the  next  filter  may  commence  as  soon  as  sufficient  data  are  available  from 
the  previous  filter.  Therefore,  with  a  moving  buffer  of  modest  size,  the  four 
filtering  operations  can  practically  be  performed  in  parallel. 

The  size  of  each  of  the  1-D  filters  shpuld  be  selected  so  that  the  1-D 
filters  may  follow  closely  the  contour  of  the  edges  in  the  image.  On  the  other 
hand,  it  should  be  sufficiently  large  for  adequate  noise  reduction. 

The  operators  7,-  ’s  are  data  dependent,  space  variant,  and,  therefore 
non-commutative.  Therefore,  the  order  in  which  the  four  1-D  filters  are  per¬ 
formed  will  affect  the  final  results.  For  a  general  image  with  no  prominent 
correlation  in  any  particular  direction,  the  different  orders  do  not  result  in  any 
significant  difference  in  performance.  However,  if  an  image  shows  a  lot  of 
correlation  in,  say,  the  vertical  direction,  then  it  is  preferable  that  the  filtering 
in  that  direction  be  carried  out  first. 

In  applying  this  approach  to  an  image  restoration  system,  one  must 
remember  that  this  approach  performs  well  because  it  implicitly  takes  into 


account  the  orientation  of  the  edges.  Therefore,  it  will  achieve  better  improve¬ 
ment  over  the  2-D  approach  if  the  latter  does  not  model  edges  adequately. 

3 S  Conclusion 

A  new  1-D  approach  to  adaptive  image  restoration  has  been  presented. 
The  approach,  motivated  by  the  ability  for  1-D  filters  to  remove  noise  along  an 
edge,  aims  to  improve  the  performance  of  some  adaptive  image  restoration  sys¬ 
tems  which  do  not  model  edges  adequately.  It  is  different  from  previous  1-D 
approaches  which  typically  approximate  2-D  filters  for  computational  reasons. 
Some  specific  examples  of  using  the  new  approach  will  be  given  in  the  next 


CHAPTER  4 


APPLICATIONS  OF  THE  ONE-DIMENSIONAL  APPROACH 


4.1  Introduction 

In  this  chapter,  three  examples  of  applying  the  1-D  approach  to  exist¬ 
ing  2-D  image  restoration  systems  will  be  presented.  The  restoration  systems 
used  are 

(1)  an  adaptive  filter  based  on  local  statistics  of  the  image  [5], 

(2)  an  adaptive  filter  based  on  the  noise  sensitivity  of  human 
vision  [6],  and 

(3)  an  adaptive  filter  using  the  spectral  subtraction  technique  [7]. 

In  all  three  examples,  an  image  /  (/i1,n2)  is  degraded  by  additive  white 
noise.  The  observed  image  g(/i1,n2)  can  be  represented  by 

S  (« 2)  =  /  (» i>*  2)  +  w  ("  i>n  2)  (43) 

where  w(nt,n2)  is  a  zero-mean  white  noise  sequence  assumed  uncorrelated  with 
/(«!>»  2)- 

The  basic  principle  of  each  algorithm  will  be  reviewed.  The  1-D 
approach  wiii  be  used  to  modify  the  algorithm,  and  the  experimental  results  of 
restoration  by  both  1-D  and  2-D  approaches  will  be  presented. 


42  Adaptive  Filter  Based  on  Local  Statistics 


A  very  simple,  but  effective  restoration  algorithm  was  proposed  by 
Lee  [5].  In  this  algorithm,  edges  are  modeled  by  higher  local  signal  variance, 
and  preserved  by  an  adaptive  filter  determined  from  the  values  of  the  local 
variance  and  mean. 


4 2.1  The  2-D  LLSE  Algorithm 

By  modeling  an  image  /  (ni,n2)  as  consisting  of  a  nonstationary  mean 
component  [18]  and  nonstationary  and  approximately  white  fluctuations  about 
the  mean,  the  linear  least  square  error  (LLSE)  estimate  of  /  (nl5n2),  based  on 
local  stationarity,  is  readily  derived  as 


f(nhn2) 


W*2) 


\g  (n i ,n2)-mg  (nlfn2)]  +  mf  (n1,n2)  (42) 


where  mf(nl,n2)  and  aj(nx,n2)  are,  respectively,  the  a  priori  local  mean  and 
variance  of  /  (n1,n2)  ,  o^(n1,n2)  is  the  local  noise  variance,  and  mg (ni,n2)  is  the 
mean  of  g{nx,n{). 

The  a  priori  mean  and  variance  of  /  ( nx,it2 )  are  estimated  from  the 
observations  in  a  (2p  +l)x(2^  +1)  window  as 

1  "\*V  "2  -h? 

(n hn 2)  =  (n  2)  =  ■  %  2  *  (* b* 2>  (43) 


and 


f  vl{n\>*i)-i*l(nx,n2)  if  d2(n1,fi2)><x;2(n1,n2) 
d/(«i,n2)=  o  otherwise 


,  i  »r*P  »:+V  . 

*/<"  i»«  2)  =  (2p  +i)(2g  +fjk  2  ^  2  k  (*  i»*  2)“**,  ("  i>»  2)f  (4-5) 

The  LLSE  algorithm  can  be  viewed  as  an  example  of  the  two-channel 
process  described  in  Figure  41.  In  the  two-channel  process,  the  input  signal  is 
divided  into  two  paths:  a  low -pass  signal  ( mg )  and  a  high-pass  signal  (g  ~mg)- 
The  high-pass  signal  is  scaled  by  a  function  of  its  variance,  as  described  by  Eqn. 
(42),  and  combined  with  the  low-pass  signal  to  form  the  output. 

The  filter  is  spatially  variant  and  nonlinear.  Its  coefficients  are  calcu¬ 
lated  at  every  pixel  with  the  window  sliding  in  the  direction  of  scan.  Edges  are 
preserved  through  the  ratio  of  aj («i,n2)  to  cr*(n i,n2).  As  cr*(nj,/i2)  varies  from 
0  to  ®,  the  filter  is  a  compromise  between  the  simple  local  averaging  filter  and 
the  identity  filter.  Thus,  at  high  SNR  regions,  such  as  edge  regions,  where 
a/(n1,n2)  is  much  higher  than  cr» (nhn2),  the  estimate  is  nearly  equal  to  the 
observation,  and  little  noise  is  removed. 

The  local  LLSE  filter  preserves  edges  by  removing  less  noise  in  the 
edge  region  than  the  flat  region.  This  is  acceptable  if  the  image  has  a  high  SNR 
and  is  processed  for  human  viewing,  as  the  human  visual  system  is  less  sensitive 
to  noise  in  a  busy  area.  However,  for  images  with  very  low  SNR,  the 
unremoved  noise  near  edges  may  become  clearly  visible.  Furthermore,  if  the 
processed  image  is  to  be  used  for  subsequent  processing,  rather  than  human 
viewing,  it  is  preferable  to  remove  noise  from  the  edge  regions  as  well.  It  will 
be  demonstrated  that  by  applying  the  1-0  approach,  one  can  remove  noise  near 
an  edge,  while  preserving  the  edge. 
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4 22  The  1-D  LLSE  Algorithm 

Consider  a  1-D  spatially  variant  filter  oriented  in  the  horizontal  direc¬ 
tion  and  implemented  using  the  same  principle  as  the  LLSE  algorithm.  The 
output  of  the  1-D  filter  is 


g/("l»"2) 

^jinx,n2)Msl{nXyn2) 


/l(»lt«2>  =  Ti7 - ; - Ti - 7  k(«l."2)-»»i(«l»«2)]  +"»/("l,«2)  (4j6) 


where  the  mean  and  variance  estimates  are  calculated  from 


1  iy 

^f(nx,n2)=rhs{nx,n2)=-^—^  X 


^lin\,n2)~^l{nx,n2)  if  aj(nhn2)>ul(nx,n2) 


®/("i,«2)  =  o 


otherwise 


where 


1  *\*v 

g/(»l»"2>  =  7ZZ  ,u  2  \8(l\,n2)~mg(nx,n2)f 
K  P  a;*.=»,-p 


The  second  1-D  filter,  say,  the  vertical  filter,  is  obtained  in  a  similar 
way  and  applied  to  / i(«i,/i2)-  We  note,  however,  that  the  noise  term  a^(nx,n2) 
should  be  updated  as  the  first  filter  has  reduced  the  noise  power.  Thus,  if 
hx(k x,nx,n2)  represents  the  unit  sample  response  of  the  first  filter  at  ( nx,n2),we 
can  estimate  the  reduced  noise  power  at  each  pixel,  (nx,n2),  as 


»i 

^wl(nbn2)=  2  hl(klinbn2)  ^(kun2) 
*i="i  ~p 


(4-10) 


Extension  to  the  two  remaining  1-D  filters  at  45  and  135  degrees  is  similar. 


The  resulting  system  is  a  cascade  of  1-D  filters  capable  of  adapting  to 
the  edge  orientation  in  the  image,  whereas  the  2-D  filter  discussed  earlier  is 
insensitive  to  edge  orientation.  Thus,  a  sharp  edge  inclined  at  a  large  angle  to 
the  filtering  direction  remains  practically  intact,  while  the  noise  at  the  edge  is 
removed  by  one  of  the  filters  oriented  closest  to  the  direction  of  the  edge.  The 
1-D  approach  also  performs  better  in  low  contrast  regions.  If  the  1-D  filters  are 
each  of  length  M  ,  the  cascaded  filters  are  in  effect  an  octagonal  filter  with 
sides  of  length  A#  and  a  filter  support  of  size  1M  z-10Af  44.  This  is  considerably 
larger  than  M 2  for  a  M  xAf  2-D  filter,  and  more  effective  in  noise  reduction. 
The  objective  and  subjective  improvement  in  performance  over  the  2-D  LLSE 
filter  will  be  shown  in  the  next  section. 


4 23  EXPERIMENTAL  RESULTS 

This  section  presents  experimental  results  where  both  2-D  and  1-D 
filtering  techniques,  discussed  in  the  previous  sections,  were  applied  to  restore 
images  degraded  by  additive  white  noise.  In  the  1-D  method,  the  image  was 
filtered  horizontally  (0°),  vertically  (90°),  along  the  (45°)  angle,  and  finally  along 
the  (135°)  angle.  The  window  sizes  were  chosen  as  5x5  and  5x1,  respectively,  in 
the  2-D  and  1-D  methods  for  the  best  tradeoff  between  noise  removal  and  reso¬ 
lution. 

Figures  42  and  43  illustrate  the  behavior  of  the  two  filters  near  an 
edge.  Figure  42(a)  shows  an  image,  consisting  of  two  flat  regions  separated  by  a 
vertical  edge.  The  image  is  degraded  by  noise,  as  shown  in  Figure  42(b).  The 
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result  of  2-D  Sitering,  as  shown  in  Figure  42(c),  indicates  practically  no  noise 
smoothing  near  the  edge.  Figure  43  shows  that  the  noise  near  the  edge  is 
reduced  by  the  vertical  1-D  filter,  while  the  edge  is  not  blurred  by  the  other 
three  filters.  Figure  43(d)  also  shows  more  noise  reduction  in  the  flat  regions 
than  that  achieved  by  the  2-D  method. 

The  results  of  processing  real  images  are  shown  in  Figures  4j6  and  4.7. 
Two  images  of  the  ’BANK’  and  the  ’GIRL’,  which  have  different  correlation 
characteristics,  are  chosen.  The  original  pictures,  each  with  256x256  8-bit  pixels, 
are  shown  in  Figures  4.4(a)  and  45(a).  The  images  were  degraded  by  additive 
white  Gaussian  noise  at  various  SNRs.  The  SNR  of  an  image,  say  g(n^,n 2),  is 
defined  as 


SNR  =  10  log 


22k(ni»a2)-/  («i»n2)]2 

n \  /ij 

221 /  (*i>*2)-”»/]2 
*1  *1 


(4-11) 


where  ntf  is  the  mean  of  the  original  image  /  (ni,n2).  Two  degraded  images  at 
a  SNR  of  10  dB  are  shown  in  Figures  4.4(b)  and  45(b).  In  both  methods,  the 
noise  variance  was  estimated  from  a  window  of  about  1500  pixels  in  a  flat  area 
of  the  degraded  image. 

The  results  are  summarized  in  Table  1,  which  lists  the  improvement  in 
SNR,  for  input  SNRs  ranging  from  0  dB  to  10  dB. 


Table  4 1:  SNR  improvement:  2-D  vs  1-D  LLSE  algorithms 


Input 

Improvement  (dB) 

SNR 

BANK’  Image 

’GIRL’  Image 

(dB) 

2-D  filter 

1-D  filter 

2-D  filter 

1-D  filter 

0 

6j64 

761 

866 

931 

3 

609 

654 

730 

814 

6 

5.46 

617 

6.79 

723 

10 

4.47 

459 

553 

5.74 

Quantitatively,  1-D  filtering  results  in  up  to  1  dB  improvement  over  2-D  filter¬ 
ing.  The  improvement  is  more  significant  for  the  ’BANK’  image,  which 
possesses  more  distinctive  correlation  in  the  four  directions  considered.  The 
improvement  in  visual  quality  can  be  seen  in  figures  4j6  and  4.7.  Figures  46(a) 
and  4.7(a)  show  the  results  of  the  2-D  method,  and  figures  4j6(b)  and  4.7(b)  show 
the  results  of  the  1-D  method.  The  2-D  filtering  preserves  edges,  but  does  so  at 
the  expense  of  insufficient  noise  removal,  particularly  in  the  edge  regions. 
Images  restored  by  the  1-D  approach  show  its  superiority  in  preserving  edges 
and  reducing  noise  in  all  regions,  including  the  edge  areas.  This  is  shown  more 
clearly  in  Figures  43  and  45,  which  are  enlarged  segments  of  Figures  4.4  -  4.7. 
These  results  indicate  that  the  1-D  filtering  is  more  effective  in  both  high  and 
low  contrast  regions. 
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Figure  43  (a)  An  enlarged  section  of  figure  44(a). 

(b)  An  enlarged  section  of  figure  44(b). 

(c)  An  enlarged  section  of  figure  4j6(a). 

(d)  An  enlarged  section  of  figure  4j6(b). 
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4J  Adaptive  Filter  Based  on  a  Noise  Visibility  Function 
•  The  sensitivity  of  the  human  visual  system  to  noise  in  an  image 

decreases  with  the  amount  of  local  spatial  details.  In  other  words,  the  sensi¬ 
tivity  is  highest  in  the  flat  areas,  and  lowest  in  the  busy  areas.  This  effect  is 
®  incorporated  in  a  restoration  system  proposed  by  Anderson  and  Netravali  [6]. 

They  quantified  the  noise  sensitivity  by  a  Visibility  function’,  which  represents 

a  subjective  tradeoff  between  resolution  and  noise.  Using  this  Visibility  func- 

9 

tion*  they  constructed  a  performance  index,  which  leads  to  an  adaptive  filter 
(called  the  S-type  filter)  capable  of  maintaining  the  tradeoff  between  the  noise 
removed  and  the  blur  introduced  by  the  filter.  This  section  describes  further 
improvement  to  this  algorithm  by  the  1-D  approach. 

9 

4.3.1  The  2-D  S-type  Filter 
The  Visibility  Function 

**  A  ’masking  function’,  Af  (n \,n2),  at  a  pixel  (n!,/*2)  is  defined  as  a  meas¬ 

ure  of  spatial  detail  at  the  pixel: 

J#(»„-.2)  =  "£  (4J2) 

*i  =*rp  =*2-? 

where  I  l(n1,n2)-(4 1,^2) 1  1  denotes  the  Euclidean  distance  between  (n1,n2)  and 
,  (khk2);  mh(k^k2)  and  mv (k lfk 2)  are,  respectively,  the  horizontal  and  vertical 

slopes  of  the  image  intensity  at  (k  i,k  2);  C  is  a  constant  (taken  as  035).  Note 
that  M  (nlfrt2)  increases  monotonically  with  the  amount  of  spatial  detail  in  a 
'  neighborhood  surrounding  (ni,/i2). 
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A  subjective  test  was  conducted  to  determine  a  Visibility  function’, 
F  (n\,n2),  which  gives  the  relative  visibility  of  a  unit  of  noise  added  to  all  parts 
in  the  image  where  M  (n1,n2)  bas  a  certain  value.  In  other  words,  the  subjective 
’noise  visibility’  at  a  point  where  F  =Fq  is  equal  to  that  at  another  point  where 
F  =  ViF o  and  the  amount  of  noise  is  doubled.  It  was  found  that  F(n1,/i2)  can 
be  approximated  by  a  decreasing  function  of  Af  («i,n2)  as 

F(nl,n2)=e-f*i{n"ni)  (4J3) 

for  some  appropriate  value  of  p. 


The  Restoration  Filter 

Let  the  impulse  response  of  the  2-D  spatial  variant  filter  be 
A(i1,i2;n1,/i2),where 


A(*i,*2;ul,/i2) 


(#0 


K=Q 


~q  —k ,—q  =S*2=S<? 


otherwise 


(4-14) 


It  is  desired  that  the  filter  removes  as  much  noise  as  possible.  On  the 
other  hand,  the  filter  must  also  introduce  little  blurring.  The  former  is  meas¬ 
ured  by  the  relative  amount  of  noise  passed,  va ,  given  by 

va(n hnz)=  i)  %  h2(ki>k2>nvn2)  (4-15) 

and  the  latter  is  quantified  by  the  spread  function 

*'a('»b'»2)  =  £  i  h2(kl,k2-,nun2)  (kf  +  k$)  (4.16) 

*1=-?  *2=~V 


Since  the  two  requirements  are  somewhat  contradictory,  the  optimal 
filter  is  obtained  by  minimizing  the  joint  objective  function 

J  (n\,n2)  =avJ(B1,«2)  +  (l-a)wa(ni,n2)  (4.17) 

subject  to  the  constraint 

%  %  k(kltk2;nhn2)  =  1  (4J8) 

kt=-q  k2=-q 

where  a  (0  < a  <  1)  is  a  tuning  parameter.  Using  the  Lagrange  multiplier  X,  the 
solution  is  given  by 


h(khk2;nhn2) 


X 


a  +(l-a)(*12  +  *|)  -q-Skx^q  ,~q^k2^q 

(4-19) 


^  0  otherwise 

where  X  is  adjusted  so  that  the  constraint  (4JL8)  is  satisfied. 


The  Role  of  F  ( ni,n2 ) 

The  filter  is  determined  so  that  in  flat  areas,  it  approaches  an  averaging 
filter,  and  in  edge  areas,  it  approaches  the  identity  filter.  This  is  achieved  by 
tuning  a  so  that  the  amount  of  Visible  noise’  is  a  constant.  More  specifically, 
let  <{>  determine  the  relative  amount  of  noise  passed  by  the  filter  in  a  perfectly 
flat  area,  where  M  (n\,n2)=0  and  F  ("i>«2)  =  T  The  value  of  4>  is  chosen  from 


the  range  between 


1 


(corresponding  to  a  (2 q  +l)x(2 q  +1)  averaging  filter) 


(2 1  +1)2 

and  1  (corresponding  to  an  identity  filter),  and  held  constant  over  the  entire 
image.  At  a  pixel  where  M  (ni,/»2)>0,  set  a  so  that  the  Visible  noise’  at  the 


va(nhn2)F't(nhn2)  =  4> 


(420) 


where  y  is  another  tuning  parameter.  The  parameter  y  is  held  constant  over 
the  entire  image,  for  more  control  on  the  response  of  the  filter  to  the  Visibility 
function’.  We  note  that  va  increases  as  M  increases. 

The  algorithm  to  determine  the  filter  is,  therefore, 

(1)  Choose  a  value  of  <j>  and  a  value  of  y. 

(2)  Determine  M  (nx,n2)  and  F(nifn2 )  by  Eqn.  (4J2)  and  Eqn. 

(4J3). 

(3)  Solve  Eqn.  (420)  for  va  (nj,^). 

(4)  Iteratively  adjust  a  and  determine  h(ki,k2‘,n\,n{)  by  Eqn. 

(4.19)  until  Eqn.  (4.15)  is  satisfied. 

(5)  Repeat  steps  (2)-(4)  for  all  (/»i,n2)  in  the  image. 

We  note  that  the  impulse  response  h{khk2',nhnj)  is  isotropic,  and 
changes  its  shape  with  the  amount  of  local  detail.  Thus,  at  points  of  high  detail, 
the  response  is  peaked,  and  in  the  fiat  area,  the  response  is  flat.  Details  in  the 
image  are  therefore  preserved,  and  the  noise  in  the  fiat  areas  is  removed. 
Although  the  algorithm  does  not  explicitly  require  knowledge  about  the  noise, 
the  value  of  y  that  gives  the  best  performance  is  influenced  by  the  amount  of 
noise  present  via  the  masking  function  M  («i,n2).  More  specifically,  for  images 
of  low  SNR’s,  the  value  of  y  will  have  to  be  tuned  down  to  achieve  better  noise 
smoothing. 


432  The  1-D  S-type  Filter 


Consider  a  1-D  filter  oriented  in  the  horizontal  direction  and  designed 
using  the  same  principle  as  the  S-type  filter.  The  impulse  response  of  the  1-D 
horizontal  S-type  filter  is  given  by 


X 

a  -t-Cl-ct)*!2 


0 


-q  <kx<q 
otherwise 


(421) 


The  procedure  to  determine  the  filter  coefficients  is  similar  to  that  in 
the  2-D  case.  For  example,  the  horizontal  masking  function  is  calculated  by 

(422) 

The  noisy  image  is  then  filtered  by  ni,n2).  The  resulting  image  is  filtered 
sequentially  in  the  other  three  directions  by  the  other  three  1-D  filters,  each 
determined  in  a  similar  manner. 


4JJ  Experimental  Results 

The  same  ’BANK’  and  ’GIRL’  images  used  in  the  previous  example  are 
used  to  illustrate  the  effectiveness  of  the  1-D  S-type  filter.  The  window  sizes 
were  again  chosen  as  5x5  and  5x1  for  the  2-D  and  1-D  algorithms,  respectively. 
The  value  of  p  in  Eqn.  (4J3)  was  chosen  as  0j02,so  that  the  visibility  function  is 
similar  to  that  given  by  [6].  The  parameter  <j>  was  chosen  as  0D4  and  02,  respec¬ 
tively,  for  the  2-D  and  1-D  algorithms.  In  both  algorithms,  the  values  of  the 


masking  function  M(n1,n2)  were  estimated  from  the  original  noiseless  image 
because  estimating  M  (n1,«2)  from  the  noisy  image  is  unreliable  (as  noted  in 
[6]),  especially  for  images  of  very  low  SNR’s.  Although  better  ways  of  estimat¬ 
ing  M  (n^n  2)  from  the  noisy  data  have  been  proposed  (see  [16],  for  example), 
noiseless  estimate  of  M{n^,n  2)  was  used  for  the  purpose  of  comparing  the  2-D 
and  1-D  algorithms.  Since  the  performance  of  the  algorithms  depends  on  the 
value  of  y  selected,  the  value  of  y  for  each  image  was  adjusted  until  the  best 
SNR  improvement  was  obtained. 

The  results  are  summarized  in  Table  4 2  for  various  input  SNRs. 


Table  4 2:  SNR  improvement:  2-D  vs  1-D  S-type  filters 


Input 

Improvement  (dB) 

SNR 

’BANK’  Image 

’GIRL’  Image 

(dB) 

2-D  filter 

1-D  filter 

2-D  filter 

1-D  filter 

0 

809 

933 

955 

1050 

3 

7.42 

833 

897 

9.79 

6 

6.43 

796 

800 

8.74 

10 

503 

6j66 

653 

111 

Similar  to  the  LLSE  algorithm  in  section  42,  the  1-D  S-type  filter  performs 
better  than  the  2-D  S-type  filter.  In  terms  of  SNR,  the  1-D  filter  results  in  0j6  to 


15  dB  improvement  over  the  2-D  filter.  The  improvement  is  again  more 
significant  for  the  ’BANK’  image.  Figures  410  and  411  show  the  results  of  res¬ 
toring  the  degraded  ’BANK’  and  ’GIRL  images  at  an  SNR  of  10  dB  (see  Figures 
4.4  and  45).  The  2-D  filter  does  not  result  in  sufficient  noise  smoothing  in  both 
the  flat  areas  and  the  edge  areas,  as  shown  in  Figures  410(a)  and  Figure  411(a). 
This  is  improved  by  the  1-D  filter,  as  shown  in  Figures  410(b)  and  411(b). 

44  Short  Space  Spectral  Subtraction  Technique 

Image  restoration  in  the  frequency  domain  faces  the  problem  of 
estimating  the  power  spectral  density, Pf  ((Dj,^),  of  the  original  image  /  ( ni,n2 ), 
from  the  noisy  observation.  Lim  [7],  using  a  particular  method  of  expressing 
Pf  (oi^o^)  as  a  function  of  /  («i,a2),  proposed  a  spectral  subtraction  scheme  for 
estimating  /  ( nlyn2 )  directly  from  g(nun2).  The  algorithm  is  implemented  on  a 
short  space  basis,  in  which  the  degraded  image  is  divided  into  many  subimages. 
Each  subimage  is  restored  separately,  and  the  subimages  are  then  combined. 

44.1  The  2-D  Short  Space  Spectral  Subtraction  Technique 

A  short  space  window  rij(nl,n2)  is  first  applied  to  g(rthn2).  The  win¬ 
dowed  observation  g4y(nt,n2)  is  related  to  the  windowed  image  /IJ(n1,n2)  and 
the  windowed  noise  w •  (n!,/i2)  by 

8ij  (n  hn 2)  =  fij  (n  i,n 2)  +  wu  (n  hn  2)  (423) 

Then,  by  subtracting  the  noise  spectrum  from  the  spectrum  of  the  noisy  obser¬ 
vation,  the  estimate  of  Fiy  (<ui,&>2)  is  given  by 


Figure  411  (a)  Figure  45(b)  restored  by  the  2-D  S-type  filter. 

(b)  Figure  45(b)  restored  by  the  1-D  S-type  filter. 
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1 1 Gij  (<oi,o>2)  I2  -  a  k  Pw (<!»!, cjj)]*4 

\F, J  («1,“»2)  I  =  if  1  Gij  (<o,,a>2)  I2  >  a  k  Pw  (w1)W2)  (424) 

'  0  otherwise 

and 

3  Ft;  (cot.o^)  =  <}  Gi;  (<*»!, W2)  (425) 

where  (01,0^)  is  the  power  spectral  density  of  the  noise,  assumed  uniform 
over  the  image.  The  parameter  a  determines  the  amount  of  subtraction,  and 
can  be  adjusted  to  give  the  best  performance.  The  constant  k  normalizes  the 
power  and  energy  spectral  density,  and  is  given  by 

*  =  XX  r.v(ni’n2)  (^26) 

*1  n2 

The  estimate  f  (n1,n2)  is  then  constructed  by  combining /i;  (n1,n2)  as  follows: 

(427) 
*  J 

The  window  r{-  (n  j,n2)  therefore  must  satisfy  the  requirement  that 

22  r,y  («i,n  2)  =  1,  for  all  (ni,/*2)  of  interest.  (428) 

*  J 

4 A2  The  1-D  Short  Space  Spectral  Subtraction  Technique 

To  use  the  same  spectral  subtraction  principle  for  the  design  of  1-D 


filters,  1-D  windows  and  1-D  spectra  are  used.  For  example,  in  the  horizontal 
filter,  a  1-D  window  r  1(y  (n^  is  used  to  window  the  data.  The  first  estimate  of 
fij(,nhn2)  is  given  by 


1 1  Gij  ("1) 1  -  « *  i  Pw  (<*>i)J  if  i Gij  («i)  i  >  «  *  r^w  (“0 

I  FvjMI  =  (429] 

0  otherwise 


3  F  l ,ij  (°>i)  ~  3  Gij  (“l) 


(430) 


where 


*1  =2  rv,(»i) 


(431) 


The  results  from  the  various  windows  are  combined  to  form  the  output  of  the 
horizontal  filter,  as  in  Eqn.  (427). 

The  second  filter,  say,  the  vertical  filter,  may  be  applied  in  a  similar 
manner.  However,  the  noise  spectrum  has  to  be  updated  because  of  the  first 
filter.  Specifically,  the  new  noise  spectrum  can  be  approximated  as 


Pwxij  («t)  =  Pw  («*»i)  lH  1  ,ij  (<*>t) 1 ' 


(432) 


where  H  x  ^  (oij)  is  the  frequency  response  of  the  first  filter  given  by 


[  IC,V  (oiQ  I2  -  a  k  |  Pw  (on)]  * 

I (taj)  I  if  I (co,)  \2  >  a  kl  Pw  (coO 

H  Uj  (wi)  ~  (433) 


otherwise 


Extension  to  the  other  two  filters  is  similar. 
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44J  Experimental  Results 

The  2-D  and  1-D  methods  were  applied  to  the  ’BANK’  and  ’GIRL’ 
images.  The  subimage  sizes  were  chosen  as  16x16  and  16x1,  respectively.  Separ¬ 
able  triangular  windows  were  used  for  the  2-D  method,  and  triangular  windows 
were  used  for  the  1-D  method.  To  simplify  calculations,  the  noise  at  the  output 
of  each  1-D  filter  was  assumed  white.  The  SNR  improvements  are  summarized 
in  Table  43.  In  obtaining  these  results,  the  subtraction  factor  a  was  adjusted  to 
obtain  the  best  SNR  improvement  for  any  particular  input  SNR.  The  values  of 
a  are  different  for  the  2-D  and  1-D  algorithms  and  range  from  1.5  for  an  input 
SNR  of  10  dB  to  4.0  for  an  input  SNR  of  OdB. 

Table  43:  SNR  improvement:  2-D  vs  1-D  spectral  subtraction  techniques 


Input 

Improvement  (dB) 

SNR 

•BANK’  Image 

’GIRL’  Image 

(dB) 

2-D  filter 

1-D  filter 

2-D  filter 

1-D  filter 

0 

165 

7  25 

9.45 

957 

3 

703 

627 

8.71 

8458 

6 

605 

5J9 

751 

736 

10 

4.73 

3S5 

599 

569 

4 


1 


<. 


J 


As  shown  in  Table  43,  the  1-D  technique  does  not  result  in  better  SNR 
than  the  2-D  technique.  Examples  of  restoration  of  images  at  an  input  SNR  of 
10  dB  are  shown  in  Figures  4 32  and  4 33.  The  results  by  2-D  spectral  subtraction 
in  Figures  4.12(a)  and  433(a)  clearly  show  its  ability  to  preserve  edges  and 
details.  Look,  for  example,  at  the  vertical  lines  on  the  wall  above  the  window 
in  the  ’BANK’  picture,  and  the  hair  in  the  ’GIRL’  picture.  This  indicates  that 
the  2-D  spectrum  as  estimated  by  the  spectral  subtraction  technique  can  model 
the  edges  very  well.  As  expected,  the  1-D  spectrum  as  estimated  by  the  1-D 
approach  does  not  contain  as  much  edge  information  as  the  2-D  spectrum,  and 
therefore,  its  ability  to  preserve  edges  is  inferior  to  the  2-D  technique.  On  the 
other  hand,  we  notice  the  presence  of  a  harmonic  pattern  in  Figures  4  J2(a)  and 
433(a).  This,  as  explained  by  [19],  is  due  to  the  presence  of  a  few  narrowband 
peaks  of  large  amplitude  in  the  residue  after  the  spectral  subtraction.  The 
amount  of  this  type  of  artifacts  is  reduced  by  the  1-D  technique,  as  shown  in 
Figures  432(b)  and  433(b). 

This  example  illustrates  a  limitation  of  the  1-D  approach,  as  explained 
in  Chapter  3.  Specifically,  if  the  2-D  approach  is  based  on  an  accurate  model  of 
the  edges,  the  1-D  approach  is  not  likely  to  improve  its  edge  preserving  ability. 

4S  Summary  and  Conclusion 

In  this  chapter,  three  existing  2-D  adaptive  image  restoration  tech¬ 
niques  were  reviewed.  In  each  case,  the  1-D  approach  was  applied,  using  the 
same  principle  as  in  the  2-D  technique.  Significant  improvement  was  achieved 
by  the  1-D  approach  in  the  first  two  examples,  namely,  the  adaptive  filter  based 


(a)  (b) 

Figure  4J2  (a)  Figure  4.4(b)  restored  by  the  2-D  spectral 

subtraction  technique. 


(b)  Figure  4.4(b)  restored  by  the  1-D  spectral 
subtraction  technique- 


(a)  (b) 


Figure  4J3  (a)  Figure  4.5(b)  restored  by  the  2-D  spectral 
subtraction  technique. 

(b)  Figure  43(b)  restored  by  the  1-D  spectral 
subtraction  technique. 


on  local  statistics  and  the  adaptive  filter  based  on  subjective  noise  visibility.  In 
these  examples,  the  edges  were  not  adequately  modeled,  and  the  2 -D  filters 
were  therefore  isotropic.  The  sequence  of  four  1-D  filters  in  four  directions 
compensated  for  this  inadequacy  in  the  model,  and  improved  the  noise  smooth¬ 
ing  capability  near  the  edges. 

In  the  third  example,  the  2-D  spectral  subtraction  technique  showed  its 
superiority  in  preserving  edges  and  details,  as  the  2-D  spectrum  could  accu¬ 
rately  describe  the  edges  and  details.  No  improvement,  measured  in  terms  of 
SNR,  was  achieved  by  the  1-D  approach.  However,  the  1-D  approach  was  shown 
to  reduce  the  amount  of  artifacts  present  in  the  images  restored  by  the  2-D 
technique. 

From  these  results,  it  can  be  concluded  that  the  1-D  approach  can 
improve  the  performance  of  some  image  restoration  systems  which  do  not 
model  edges  adequately.  The  improvement  is  achieved  by  better  noise  reduc¬ 
tion  in  the  flat  regions  as  well  as  in  the  edge  regions,  while  the  resolution  of 
the  image  is  not  sacrificed. 
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CHAPTER  5 

FURTHER  APPLICATIONS  OF  THE  ONE-DIMENSIONAL  APPROACH 


6 


5.1  Introduction 

The  1-D  approach  developed  in  Chapter  3  is  suitable  for  restoring 
images  degraded  by  additive  noise.  Specific  examples  were  shown  in  the  previ¬ 
ous  chapter  to  illustrate  its  effectiveness.  This  chapter  deals  with  the  applica¬ 
tion  of  the  restoration  techniques  based  on  the  1-D  approach  to  three  other 
specific  restoration  problems,  namely, 

(1)  the  restoration  of  images  degraded  by  blurring  as  well  as  addi¬ 
tive  noise, 

(2)  the  restoration  of  images  degraded  by  multiplicative  noise,  and 

(3)  the  reduction  of  quantization  noise  in  pulse  code  modulation 
image  coding. 

One  approach  to  solving  these  problems  is  to  transform  the  specific 
problem  into  one  which  is  suitable  for  filtering  techniques  designed  for  reduc¬ 
ing  additive  noise.  In  each  of  these  applications,  a  noise  reduction  system  for 
additive  noise  is  incorporated  as  a  part  of  the  overall  system.  In  particular,  the 
1-D  LLSE  algorithm  developed  in  section  42  will  be  used,  because  it  is  effec¬ 
tive  in  reducing  noise  without  sacrificing  the  resolution  of  the  image. 

In  subsequent  sections,  each  of  the  applications  will  be  examined,  and 


the  experimental  results  presented. 


5.2  Restoration  of  Blurred  and  Noisy  Images 


5.2.1  The  Principle  of  Inverse  Filtering 

If  an  image  is  blurred  by  an  imaging  system,  which  is  characterized  by 
its  point  spread  function  A(n1,«2),  the  image  formation  can  be  described  by 

G  ((1)1,102)  =  H  (o>t,o>2)  F  (&>i,(i>2)  (5.1) 


One  simple  way  to  estimate  the  original  image  /  (n1,n2)  is  to  inverse  filter 
g(nx,n2),i&n 


H  (<i>i,a»2) 


(52) 


provided  H  (ai^a^)  has  no  singularity  for  all  frequencies  (wj,^).  However,  in 
the  presence  of  additive  noise  w(n1,n2),  as  shown  in  Figure  5.1(a),  the  estimate 
is  given  by 

.  W  (oji.uh) 

F  (011,0^)  =  F  (a)!, d^)  +  — - -  (53) 

H  (wt.o^) 

As  H  (cujjti^)  becomes  small  at  certain  spatial  frequencies,  its  inverse  becomes 
large,  while  the  noise  ^(0^,0^)  may  not  be  negligible  at  these  frequencies.  This 
results  in  an  amplification  of  the  noise,  and  the  restored  image  is  generally 
unsatisfactory.  Therefore,  inverse  filtering  works  well  only  if  the  SNR  of  the 
degraded  image  is  high. 

One  approach  to  overcoming  this  difficulty  is  to  preprocess  the  image 
by  a  noise  reduction  system  before  performing  inverse  filtering  [7],  as  shown  in 
Figure  5.1(b).  This  technique  would  increase  the  SNR  in  order  that  the  deblur¬ 
ring  would  perform  better.  One  desired  property  of  the  noise  reduction  system 


1 


Figure  51  (b)  A  deblurring  system  using  inverse  filter. 


-76  - 


is  that  it  must  have  good  noise  reduction  and  edge  preserving  capabilities,  in 
order  not  to  introduce  additional  blurring  to  the  image.  Based  on  the  results 
presented  in  the  last  chapter,  the  1-0  LLSE  algorithm  is  selected. 

5.2.2  Experimental  Results 

The  ’BANK’  and  ’GIRL’  images  shown  in  Figures  4 2  and  43  were 
blurred  by  a  Gaussian  shape  point-spread  function  with  standard  deviation  a 
equal  to  1  pixel.  White  Gaussian  noise  was  then  added  to  the  blurred  image. 
Examples  of  the  degraded  images  at  a  blurred-signal  to  noise  ratio  (BSNR)  of 
20  dB  are  shown  in  Figure  52.  Inverse  filtering,  both  with  and  without  the 
noise  reduction  system,  was  carried  out.  In  implementing  the  inverse  filter,  it  is 
necessary  to  set  H  (cu^o^)  to  a  minimum  value  if  H  falls  below  a  cer¬ 

tain  level,  determined  subjectively  to  give  the  best  visual  result.  This  is  to  avoid 
any  singularity  or  near  singularity  that  H  (oit,a>2)  may  have.  The  results  of 
inverse  filtering  without  noise  reduction  are  shown  in  Figures  53(a)  and  (b). 
Although  inverse  filtering  sharpens  the  image,  it  also  introduces  additional 
noise  to  the  image.  The  results  of  noise  reduction  before  inverse  filtering  are 
shown  in  Figure  53(c)  and  (d).  Much  better  improvement  in  image  quality  can 
be  seen.  These  resulu  are  consistent  with  those  reported  in  [7], 


53  Restoration  of  Images  Degraded  by  Multiplicative  Noise 


Figure  52  (a)  Figure  4.4(a)  blurred  by  a  Gaussian  PSF  and 
degraded  by  additive  noise  at  BSNR  =  20dB. 
(b)  Figure  45(a)  blurred  by  a  Gaussian  PSF  and 
degraded  by  additive  noise  at  BSNR  =  20dB. 


Figure  53  (a)  Figure  52(a)  restored  by  inverse  filter. 

(b)  Figure  52(b)  restored  by  inverse  filter. 

(c)  Figure  52(a)  restored  by  noise  reduction 
followed  by  inverse  filter. 

(d)  Figure  52(b)  restored  by  noise  reduction 


followed  by  inverse  filter. 


5  J.l  Filtering  of  Speckle  Degraded  Images  in  the  Density  Domain 

Some  physical  noise  processes  are  signal-dependent.  In  many  instances, 
the  signal-dependency  may  be  modeled  by  a  multiplicative  process.  By  taking 
logarithm  of  the  image  intensity,  one  can  transform  the  image  into  the  density 
domain,  and  convert  the  multiplicative  noise  into  an  additive  one.  Thus,  filter¬ 
ing  techniques,  developed  for  removing  additive  noise,  may  be  applied  in  the 
density  domain,  and  the  result  converted  back  into  the  intensity  domain  by 
exponentiation. 

One  particular  example  that  will  be  considered  here  is  that  of  speckle 
noise  in  images  generated  by  highly  coherent  sources,  such  as  laser  light.  It  was 
shown  [8]  that  speckle  noise  may  be  modeled  as  a  signal-independent  multipli¬ 
cative  noise.  The  degraded  image  can  thus  be  written  as 

g(ni,n2)  =f(nltn2)w(nhn2)  (5.4) 

where  w(nl,n2)  is  a  signal-independent  white  noise  sequence  whose  probability 
density  function  may  be  modeled  by 

e~w  w  >0 

)  ~  o  otherwise  (^) 

In  the  density  domain,  Eqn.  (5.4)  becomes 

logk('»n«2)l  =1°g[/‘  ("l^z)!  +  log[w'(n,,n2)]  (5j6) 

Several  techniques  for  reducing  speckle  noise  were  studied  by  Lim  and 

Nawab  [8].  The  study  indicates  that  if  only  one  frame  of  the  degraded  image  is 

available  for  processing,  the  results  are  rather  poor  due  to  the  very  low  SNR  of 
the  image.  However,  if  N  frames  of  the  same  image  degraded  by  independent 


speckle  noise  are  available,  averaging  the  N  frames,  followed  by  a  good  filter¬ 
ing  technique  in  the  density  domain,  produces  much  better  results.  The  overall 
system  for  restoring  speckle  noise  degraded  images  by  such  a  scheme  is  shown 
in  Figure  5.4.  In  [8],  the  short  space  spectral  subtraction  technique  was  proposed 
for  the  noise  reduction  system.  An  alternative  that  may  be  used  is  the  1-D 
LLSE  algorithm.  Experimental  results  of  the  system  using  the  1-D  LLSE  algo¬ 
rithm  are  shown  in  the  next  section. 

5JJ  Experimental  Results 

The  ’BANK’  and  ’GIRL’  images  in  Figures  43  and  4.4  were  degraded 
according  to  Eqn.  (5.4)  by  a  noise  sequence  whose  PDF  is  given  by  Eqn.  (55). 
Examples  of  the  degraded  images  are  shown  in  Figures  55(a)  and  (b).  Figures 
55(c)  and  (d)  show  the  results  of  averaging  eight  frames  of  independently 
degraded  images.  These  degraded  images  were  restored  in  the  density  domain 
by  the  1-D  LLSE  algorithm.  In  implementing  this  algorithm,  the  variance  of 

■n2  1 

log[w  («!,«->)]  was  taken  as  —  for  the  single-frame  case,  and  —  for  the  N  - 

o  N 

frame  case.  The  results  are  shown  in  Figure  5j6.  For  the  single-frame  case, 
although  the  speckle  noise  is  significantly  reduced,  the  quality  of  the  image  is 
still  poor.  For  the  N  -frame  case,  better  noise  reduction  and  preservation  of 
details  can  be  seen. 
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5.4  An  Application  to  Noise  Reduction  in  Image  Coding 


5.4.1  A  Quantization  Noise  Reduction  Scheme 

One  major  problem  in  image  coding  by  pulse  code  modulation  (PCM) 
techniques  is  the  quantization  noise  that  arises  from  representing  the  continu¬ 
ous  tone  of  an  image  by  a  finite  number  of  intensity  levels.  If  the  number  of 
bits  per  pixel  is  reduced  to  less  than  4,  the  ’staircase’  or  contouring  effect  of 
quantization  noise  becomes  very  objectionable.  To  achieve  better  image  quality 
in  a  low  bit  rate  (eg,  3  bits  per  pixel)  PCM  system,  Roberts  [9]  proposed  a  tech¬ 
nique  which  transforms  the  contouring  effect  into  a  less  objectionable  signal- 
independent  random  noise.  In  his  technique,  shown  in  Figure  5.7(a),  a  pseu¬ 
dorandom  noise  sequence  is  added  to  the  image  before  quantization.  The  noise 
sequence,  w(nltn 2),  is  generated  by  the  probability  density  function  given  by 


fl 


to 


I  w  I  ==4 
2 


otherwise 


(5.7) 


where  A  is  the  quantization  level.  At  the  receiver,  an  exact  replica  of  H'(n1,n2) 
is  subtracted  from  the  image.  Although  the  noise  level  in  the  resulting  image  is 
higher,  it  is  more  tolerable  than  the  quantization  noise. 

One  way  to  further  improve  Roberts’  scheme  was  proposed  by  Lim 
[10].  In  his  system,  shown  in  Figure  5.7(b),  a  noise  reduction  system  was  cas¬ 
caded  to  Roberts’  system  at  the  receiver  to  reduce  the  amount  of  the  added 
random  noise.  Significant  improvement  in  image  quality  was  reported.  In  this 
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section,  the  1-D  LLSE  algorithm  is  proposed  for  the  noise  reduction  system. 


5.4.2  Experimental  Results 

The  ’BANK’  and  ’GIRL’  images  in  Figures  43(a)  and  4.4(a)  were  quan¬ 
tized  to  3  bits  per  pixel,  as  shown  in  Figures  53(a)  and  53(a).  In  these  pictures, 
the  contouring  effect  can  be  seen  quite  clearly.  Roberts’  method  was  imple¬ 
mented,  and  the  results  are  shown  in  Figures  53(b)  and  53(b).  Although  the 
contouring  effect  is  eliminated,  the  presence  of  random  noise  in  the  images  is 
quite  noticeable.  The  1-D  LLSE  algorithm  was  then  applied  to  Figures  53(b) 
and  53(b).  In  this  algorithm,  the  noise  variance  was  calculated  from  Eqn.  (5.7) 
A2 

as  — .  The  results,  presented  in  Figures  53(c)  and  53(c),  show  that  much  of  the 

random  noise  is  effectively  removed  while  the  image  resolution  is  maintained. 
The  experiments  were  repeated  for  a  2-bit  PCM  system.  The  corresponding 
images  are  shown  in  Figures  5JL0  and  511. 

Table  53  lists  the  the  normalized  mean-square-error  (NMSE)  achieved 
by  the  two  quantization  noise  reduction  schemes  in  comparison  with  the  stan¬ 
dard  PCM  system.  The  NMSE  (in  %)  of  an  image,  say, r(nl,/i2),  with  respect  to 
the  original  image  /  (n1,n2),  is  defined  [10]  as 


22lr'(,*i»*2 )”/  \fn 2>J2 


NMSE  =100 


XXL/”  (rtl>n2)-«/]2 


where  mj  is  the  mean  of  /  (nitn2),  and 


r\nx,n2)  =a  r(nhn2)  +b 


(C) 


(b) 


Figure  5 9  3-bit  PCM  ’GIRL’  image. 

(a)  Standard  PCM. 

(b)  Roberts’  pseudonoise  technique. 

(c)  The  improved  technique. 
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with  a  and  b  chosen  such  that  /  (/»i,«2)  and  r'(nbn2)  have  the  same  mean  and 
variance.  As  seen  in  Table  51,  the  NMSE’s  of  images  processed  by  Roberts’ 
method  are  significantly  lowered  by  the  1-D  LLSE  nose  reduction  algorithm. 
These  results  are  consistent  with  the  results  reported  in  [10]. 

Table  51:  Quantization  noise  reduction  -  normalized  mean  square  error  (%) 


Image 

no.  of 

bits 

per  pixel 

standard 

PCM 

coding 

Roberts’ 

pseudonoise 

method 

Roberts’ 

method  with 

1-D  LLSE 

BANK 

2 

859 

1061 

392 

BANK 

3 

219 

250 

119 

GIRL 

2 

1053 

1723 

459 

GIRL 

3 

298 

355 

156 

5.5  Conclusion 

Three  specific  applications  of  a  restoration  technique  based  on  the  1-D 
approach  were  shown.  In  each  application,  the  problem  was  formulated  such 
that  a  part  of  the  overall  restoration  system  made  use  of  the  additive  noise 
reduction  scheme.  In  the  first  application,  the  1-D  technique  reduced  the  insta¬ 
bility  of  inverse  filter  in  a  deblurring  system.  In  the  second  application  which 
dealt  with  multiplicative  noise,  the  1-D  technique  was  shown  to  be  equally 
effective  in  the  density  domain.  In  the  third  application,  the  1-D  technique 
improved  the  Roberts’  quantization  noise  reduction  technique  by  reducing  the 


0 


amount  of  the  added  pseudorandom  noiue  in  the  image.  These  results  are  simi¬ 
lar  to  previous  results  reported  in  [7340]-  These  examples  illustrate  the  useful¬ 
ness  of  the  1-D  approach  for  other  restoration  problems  in  addition  to  those 
involving  additive  noise,  as  shown  in  the  last  chapter. 
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CHAPTER  6 


SUMMARY  AND  CONCLUSION 


6.1  Summary 

In  this  thesis,  a  new  1-D  approach  to  adaptive  image  restoration  is 
presented.  The  objective  of  this  approach  is  to  improve  the  performance  of 
the  more  general  2-D  approach  for  some  adaptive  image  restoration  systems 
which  do  not  model  edges  adequately.  More  specifically,  techniques  based  on 
the  new  approach  remove  noise  more  effectively  in  the  edge  as  well  as  non¬ 
edge  regions,  while  preserving  the  resolution  of  the  image. 

A  brief  review  of  both  nonadaptive  and  adaptive  image  restoration 
were  presented  in  Chapter  2,  with  specific  examples  to  illustrate  their  general 
principles.  Examples  were  also  presented  to  show  the  various  ways  to  approxi¬ 
mate  the  more  general  2-D  approach  by  1-D  filters.  Such  approximations  were 
used  partly  because  of  the  difficulty  in  obtaining  a  good  accurate  model  of  the 
edge,  and  partly  to  reduce  computations. 

Based  on  the  review  in  Chapter  2,  the  new  1-D  approach  was  presented 
in  Chapter  3.  This  approach  was  motivated  by  the  ability  of  1-D  filters  to 
smooth  noise  close  to  an  edge,  and  the  ability  of  adaptive  filters  to  preserve 
edges.  The  general  form  of  restoration  techniques  based  on  this  approach  is 
shown  in  Figure  3.1(b),  where  four  1-D  filters  are  cascaded.  Each  of  the  1-D 
filters  is  designed  based  on  the  same  principle  as  the  more  general  2-D  adaptive 
restoration  principle,  but  is  oriented  respectively  in  the  four  major  correlation 
directions  of  the  image,  namely,  0,  90,  45,  and  135  degrees.  The  approach  was 
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compared  with  other  1-D  approaches  which  approximate  the  2-D  approach. 

Examples  were  presented  to  demonstrate  the  effectiveness  of  the 
approach.  Specifically,  the  1-0  LLSE  algorithm  and  the  1-D  S-type  filter,  each 
developed  respectively  from  their  2-D  counterparts  [5,6],  performed  better  than 
the  2-D  approaches  in  their  ability  to  reduce  noise  and  maintain  resolution.  In 
another  example,  the  1-D  spectral  subtraction  method  did  not  perform  better 
than  the  2-D  approach  [7],  due  to  the  superiority  of  the  latter  in  modeling 
edges.  However,  the  1-D  technique  reduced  the  amount  of  harmonic  patterns 
observed  in  images  restored  by  the  2-D  technique. 

Techniques  based  on  the  1-D  approach  may  also  be  used  in  conjunction 
with  the  inverse  filter  to  restore  blurred  and  noisy  images.  Results  presented  in 
Chapter  5  showed  that  by  performing  noise  reduction  prior  to  inverse  filtering, 
the  ill-conditioning  of  inverse  filtering  could  be  reduced.  Transforming  an 
image  into  the  density  domain  enabled  the  1-D  technique  to  be  used  to  restore 
images  degraded  by  multiplicative  noise,  such  as  speckle  noise.  Since  the  SNRs 
of  speckle  images  were  typically  very  low,  it  was  necessary  to  average  several 
independently  degraded  images  before  filtering  to  obtain  better  results.  In 
another  application,  the  1-D  technique  improved  the  Roberts’  pseudonoise  tech¬ 
nique  for  quantization  noise  reduction.  The  results  obtained  in  these  applica¬ 
tions  are  similar  to  those  obtained  using  the  2-D  short  space  spectral  subtrac¬ 
tion  technique  [7340]- 


62  Suggestions  for  Future  Woru 

Although  the  1-D  approach  presented  in  this  thesis  is  based  on  a  simple 
and  somewhat  heuristic  principle,  the  results  obtained  are  very  encouraging. 
Several  refinements  and  extensions  of  the  present  work  are  suggested  in  this 
section. 

In  the  1-D  algorithms  presented  in  Chapter  4,  it  was  necessary  to 
assume  that  remaining  noise  after  every  1-D  filtering  is  white,  in  order  to  sim¬ 
plify  calculation.  This  was  only  an  approximation,  as  each  filter  not  only 
reduced  the  amount  of  noise,  but  also  introduced  spatial  correlation  to  the 
noise,  and  correlation  between  the  image  and  the  noise.  In  some  cases,  such  as 
the  LLSE  algorithm,  it  is  difficult  for  the  algorithm  to  incorporate  colored 
noise.  In  others,  such  as  the  spectral  subtraction  technique,  colored  noise  is 
allowed,  and  therefore  the  approximate  noise  spectrum  may  be  calculated  and 
incorporated  into  the  algorithm.  Although  this  will  increase  the  amount  of  cal¬ 
culation,  it  may  result  in  further  performance  improvement  for  the  1-D  tech¬ 
niques. 

The  1-D  approach  presented  is  designed  for  the  restoration  of  an  image 
frame  by  frame.  The  same  approach  may  be  extended  to  filtering  motion  pic¬ 
tures  where,  in  addition  to  the  horizontal  and  vertical  axes,  the  temporal  axis  is 
introduced.  In  this  three-dimensional  system,  the  number  of  1-D  filters  would 
become  13,  instead  of  4. 

In  the  examples  presented  in  this  thesis,  the  techniques  based  on  the 
proposed  1-D  approach  were  designed  using  the  same  principle  as  their  2-D 
counterparts.  There  is  no  reason  why  the  same  principles  of  the  existing  2-D 


methods  have  to  be  used.  For  example,  the  LLSE  algorithm  can  be  viewed  as 
an  example  of  a  two-channel  process  as  shown  in  Figure  4.1.  In  this  process,  the 
high  pass  signal  is  scaled  by  a  particular  function  of  its  variance.  One 
modification  of  the  1-D  LLSE  algorithm  may  be  to  select  some  other  nonlinear 
functions  to  further  improve  the  edge  preserving  capability  of  the  1-D  algo¬ 
rithm. 

As  a  further  extension,  an  entirely  new  restoration  algorithm  might  be 
developed  based  on  the  general  principle  of  the  proposed  1-D  approach.  The 
algorithm  would  then  be  implemented  as  four  1-D  filters,  each  not  necessarily 
based  on  any  existing  restoration  principle.  The  development  of  each  1-D  filter 
would  be  based  only  on  1-D  design  techniques,  thus  avoiding  problems  that 
might  be  encountered  in  designing  2-D  filters. 

63  Conclusion 

Based  on  the  examples  shown  in  this  thesis,  we  conclude  that  the  1-D 
approach  developed  in  this  thesis  can  improve  the  performance  of  the  more 
general  2-D  techniques  for  some  adaptive  image  restoration  systems.  Although 
this  is  not  a  general  approach  that  can  be  applied  to  the  majority  of  existing 
image  restoration  schemes,  it  has  potential  to  be  useful  in  developing  new  res¬ 
toration  techniques. 
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